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Highlights 

Highlights  of  GAO-15-627,  a  report  to 
congressional  committees. 


DOD  BUSINESS  SYSTEMS  MODERNIZATION 

Additional  Action  Needed  to  Achieve  Intended 
Outcomes 


Why  GAO  Did  This  Study 

GAO  designated  DOD’s  multibillion 
dollar  business  systems  modernization 
program  as  high  risk  in  1995,  and 
since  then  has  provided  a  series  of 
recommendations  aimed  at 
strengthening  its  institutionai  approach 
to  modernizing  its  business  systems 
investments.  Section  332  of  the  NDAA 
for  fiscal  year  2005,  as  amended, 
requires  the  department  to  take 
specific  actions  consistent  with  GAO’s 
prior  recommendations  and  included  a 
provision  for  GAO  to  review  DOD’s 
efforts,  in  addition,  the  Senate  Armed 
Services  Committee  Report  for  the 
NDAA  for  fiscal  year  201 5  included  a 
provision  for  GAO  to  evaluate  the 
usefulness  and  effectiveness  of  DOD’s 
business  enterprise  architecture  and 
business  process  reengineering 
processes.  This  report  addresses  both 
of  those  provisions. 

in  evaluating  the  department’s 
compliance,  GAO  analyzed  DOD’s 
efforts  to  address  open 
recommendations  made  in  previous 
reviews.  To  evaluate  the  usefulness 
and  effectiveness  of  the  department’s 
business  enterprise  architecture  and 
business  process  reengineering 
processes,  GAO  surveyed  the  military 
department  portfoiio  managers  (24  in 
total)  and  interviewed  officials.  The 
response  rate  for  the  survey  was  100 
percent,  making  the  results  of  the 
survey  generalizable. 

What  GAO  Recommends 

GAO  recommends  that  DOD  utilize  the 
resuits  of  the  survey  to  determine 
additional  actions  that  can  improve 
management  of  its  business  process 
reengineering  and  enterprise 
architecture  activities.  DOD  concurred 
with  the  recommendation. 

View  GAO-15-627.  For  more  information, 
contact  Carol  R.  Cha  at  (202)  512-4456  or 
chac@gao.gov. 


What  GAO  Found 

The  Department  of  Defense  (DOD)  has  impiemented  5  of  the  16 
recommendations  made  by  GAO  since  June  201 1  to  address  each  of  the 
overarching  provisions  for  improving  business  systems  management  in  the 
Ronald  W.  Reagan  National  Defense  Authorization  Act  for  Fiscal  Year  2005,  as 
amended  (NDAA)  (10  U.S.C.  §  2222)  (see  tabie).  For  exampie,  it  has 
impiemented  the  recommendation  to  improve  the  data  avaiiabie  for  its  business 
systems  by  making  improvements  to  its  repositories  used  for  tracking  information 
about  the  systems.  Based  on  GAO’s  anaiysis,  the  department  has  partiaiiy 
impiemented  the  remaining  11  recommendations.  Impiementing  aii 
recommended  actions  wiii  improve  DOD’s  modernization  management  controis 
and  heip  fuifiii  the  department’s  execution  of  the  act’s  requirements. 


Status  of  DOD’s  Implementation  of  Prior  Recommendations  Associated  with  the 
Fiscal  Year  2005  NDAA,  as  Amended,  as  of  April  2015 


NDAA  Provision 

Implemented 

Partially 

Implemented 

Total 

Business  Enterprise  Architecture 

1 

2 

3 

Enterprise  Transition  Plan 

- 

1 

1 

Investment  Management 

1 

3 

4 

Investment  Certification  and 

Approval 

2 

3 

5 

Mandated  Budgetary  Reporting 

1 

- 

1 

Other  (Human  Capital) 

- 

2 

2 

Total 

5 

11 

16 

Source:  GAO  analysis  of  DOD  documentation. 

1  GAO-15-627. 

DOD’s  business  enterprise  architecture  and  process  reengineering  efforts  are  not 
fuiiy  achieving  the  intended  outcomes  described  in  statute.  More  specificaiiy, 
portfoiio  managers  reported  through  GAO’s  survey  that  the  architecture  was  not 
effective  in  constraining  system  investments  or  enabiing  DOD  to  produce  reiiabie 
and  timeiy  information  for  decision-making  purposes,  among  other  things.  As  a 
resuit,  the  architecture  has  produced  iimited  vaiue.  Portfoiio  managers  reported 
that  the  department’s  business  process  reengineering  efforts  were  moderateiy 
effective  in  streamiining  business  processes,  but  much  iess  so  in  iimiting  the 
taiioring  of  commerciai  off-the-sheif  systems.  They  aiso  reported  that  these 
efforts  have  been  usefui  in  reaiizing  seiected  benefits,  such  as  improved 
documentation  of  business  needs. 

Managers  GAO  surveyed  reported  various  chaiienges  that  impede  the 
department’s  abiiity  to  fuiiy  achieve  intended  outcomes,  such  as  cuiturai 
resistance  to  change  and  the  iack  of  skiiied  staff.  The  department  has  work  under 
way  to  address  some  of  these  chaiienges;  however,  gaps  exist  and  the  portfoiio 
managers  provided  suggestions  on  how  to  ciose  some  of  them.  More  fuiiy 
addressing  the  chaiienges  cited  by  the  portfoiio  managers  wouid  heip  the 
department  achieve  better  outcomes,  inciuding  greater  operationai  efficiencies 
and  cost  savings. 
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July  16,  2015 

Congressional  Committees 

For  decades,  the  Department  of  Defense  (DOD)  has  been  challenged  in 
modernizing  its  business  systems.  Since  the  release  of  GAO’s  1995  High 
Risk  report,  we  have  designated  the  department’s  business  systems 
modernization  program  as  high  risk  because  of  its  vulnerability  to  fraud, 
waste,  abuse,  and  mismanagement,  and  because  of  missed  opportunities 
to  achieve  greater  efficiencies.  It  continues  to  be  on  the  high-risk  list 
today.''  In  addition,  we  have  reported^  numerous  times  that  significant 
potential  exists  for  identifying  and  avoiding  costs  associated  with 
duplicative  functionality  across  the  department’s  portfolio  of  defense 
business  systems  environment.  According  to  DOD,  this  environment 
includes  2,179  business  system  investments  and  costs  the  department 
approximately  $10  billion  in  fiscal  year  2015. 

Since  1995,  we  have  made  a  series  of  recommendations  aimed  at 
strengthening  DOD’s  institutional  approach  to  business  systems 
modernization  and  reducing  the  risks  associated  with  key  investments.^ 
Further,  Congress  included  provisions  in  the  Ronald  W.  Reagan  National 
Defense  Authorization  Act  for  Fiscal  Year  2005  (NDAA),  as  amended,'' 


^GAO,  High-Risk  Series:  An  Update,  GAO-1 5-290  (Washington,  D.C.:  Feb.  11, 2015). 

^Foilow-up  on  2011  Report:  Status  of  Actions  Taken  to  Reduce  Dupiication,  Overlap,  and 
Fragmentation,  Save  Tax  Dollars,  and  Enhance  Revenue,  GAO-1 2-453SP  (Washington, 
D.C.:  Feb.  28,  2012);  Defense  Business  Systems:  Further  Refinements  Needed  to  Guide 
the  Investment  Management  Process,  GAO-14-486  (Washington,  D.C.:  May  12,  2014). 

■^See,  for  example,  GAO,  Business  Systems  Modernization:  Summary  of  GAO’s 
Assessment  of  the  Department  of  Defense’s  Initial  Business  Enterprise  Architecture, 
GAO-03-877R  (Washington,  D.C.:  July  7,  2003);  Information  Technology:  Architecture 
Needed  to  Guide  Modernization  of  DOD’s  Financial  Operations,  GAO-01-525 
(Washington,  D.C.:  May  17,  2001);  and  High-Risk  Series:  An  Overview,  GAO/HR-95-1 
(Washington,  D.C.:  February  1995). 

"Pub.  L.  No.  1 08-375,  §  332,  1 1 8  Stat.  1 81 1 ,  1 851  -1 856  (2004),  as  amended  by  the 
NDAA  for  Fiscal  Year  2015,  Pub.  L.  No.  113-291,  §  901,  128  Stat.  3292  (2014),  the  NDAA 
for  Fiscal  Year  2014,  Pub.  L.  No.  113-66,  §  901,  127  Stat.  672,  815  (2013),  the  NDAA  for 
Fiscal  Year  201 3,  Pub.  L.  No.  112-239,  §  906,  126  Stat.  1632,  1869  (2012),  the  NDAA  for 
Fiscal  Year  2012,  Pub.  L.  No.  112-81,  §  901,  125  Stat.  1298,  1527  (2011),  and  the  NDAA 
for  Fiscal  Year  2010,  Pub.  L.  No.  111-84,  §  1072,  123  Stat.  2190,  2470  (2009).  This 
provision  is  codified  in  part  at  10  U.S.C.  §  2222. 
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that  were  consistent  with  our  past  recommendations.  Among  other  things, 
section  332  of  the  act  addresses  DOD’s  business  system  investment 
review  process,  requirements  for  certifying  and  approving  business 
systems,  and  calls  for  the  development  of  a  business  enterprise 
architecture  and  associated  enterprise  transition  plan.  The  act  also  directs 
the  Secretary  of  Defense  to  submit  an  annual  report  to  congressional 
defense  committees  on  DOD’s  compliance  with  certain  requirements  of 
the  NDAA  for  fiscal  year  2005  not  later  than  March  15  of  each  year, 
through  2016,  and  included  a  provision  for  us  to  report  on  an  assessment 
of  the  department’s  actions  to  comply  with  the  requirements  of  the  act.  In 
addition,  the  Senate  Armed  Services  Committee  report  associated  with 
the  fiscal  year  201 5  NDAA  called  for  us  to  examine  the  utility  and 
effectiveness  of  DOD’s  business  enterprise  architecture  and  business 
process  reengineering  processes.® 

Accordingly,  our  objectives  were  to  (1)  assess  the  actions  DOD  has  taken 
to  comply  with  section  332  of  the  NDAA  for  fiscal  year  2005,  as  amended, 
and  (2)  determine  the  usefulness  and  effectiveness  of  DOD’s  business 
enterprise  architecture  and  business  process  reengineering  processes. 

To  address  the  first  objective,  we  reviewed  DOD  documentation  and 
interviewed  department  officials  to  examine  DOD’s  actions  to  implement 
open  recommendations  from  previous  annual  GAO  reports  that  address 
compliance  with  the  act’s  requirements. 

For  the  second  objective,  we  analyzed  the  fiscal  year  2005  NDAA  and 
related  DOD  guidance  to  determine  intended  outcomes  of  the  business 
enterprise  architecture  and  business  process  reengineering  processes. 
We  then  developed  a  structured  data  collection  instrument  (survey)  based 
on  this  analysis  to  obtain  information  from  military  department  portfolio 
managers  on  the  extent  to  which  DOD  has  achieved  intended  business 
process  reengineering  and  business  enterprise  architecture  outcomes,  as 
well  as  information  such  as  benefits  achieved,  challenges,  and 
improvement  suggestions.  We  surveyed  these  military  department 
portfolio  managers  (24  in  total)  and  received  responses  from  all  of  them. 
Accordingly,  the  survey  results  are  generalizable.  We  then  analyzed  the 
results  to  determine  staff  perspectives  on  the  usefulness  and 


®S.  Rep.  No.  113-176,  at  147(2014),  accompanying  S.  2410  and  associated  with  H.R. 
3979  that  became  Pub.  L.  No.  1 1 3-291 . 


Page  2 


GAO-15-627  DOD  Business  Systems  Modernization 


effectiveness  of  DOD’s  processes.  We  also  met  with  representatives  from 
selected  DOD  business  system  programs  and  cognizant  DOD  officials  to 
discuss  their  perspectives  and  relevant  improvement  efforts  that  are 
planned  and  underway. 

We  conducted  this  performance  audit  from  October  2014  to  July  2015,  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objective.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objective.  Additional  details  on  our 
objectives,  scope,  and  methodology  are  contained  in  appendix  I. 


Background 


DOD  is  one  of  the  largest  and  most  complex  organizations  in  the  world.  In 
support  of  its  military  operations,  the  department  performs  an  assortment 
of  interrelated  and  interdependent  business  functions,  including  logistics 
management,  procurement,  health  care  management,  and  financial 
management.  As  we  have  previously  reported,®  the  DOD  systems 
environment  that  supports  these  business  functions  is  complex  and  error 
prone,  and  is  characterized  by  (1)  little  standardization  across  the 
department,  (2)  multiple  systems  performing  the  same  tasks,  (3)  the 
same  data  stored  in  multiple  systems,  and  (4)  the  need  for  data  to  be 
manually  entered. 

For  fiscal  year  201 5,  the  department  requested  about  $10  billion  for  its 
business  system  investments.  According  to  the  department,  as  of  April 
2015,  its  environment  includes  approximately  2,179  business  systems.  Of 
these  systems,  DOD  reports  that,  for  fiscal  year  2015,  the  department 
approved  certification  requests  for  1,182  business  systems  covered  by 
the  fiscal  year  2005  NDAA’s  certification  and  approval  requirements. 
Figure  1  shows  how  many  of  these  1,182  covered  systems  are 
associated  with  each  functional  area. 


®See,  for  example,  GAO,  DOD  Financial  Management:  Implementation  Weaknesses  in 
Army  and  Air  Force  Business  Systems  Could  Jeopardize  DOD’s  Auditability  Goals, 
GAO-12-134  (Washington,  D.C.:  Feb.  28,  2012). 
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Figure  1:  Department  of  Defense  Business  Systems  Certified  and  Approved  by 
Functionai  Area 


0.08% 


Enterprise  Information  Technoiogy 
Infrastructure  (1) 

Security  Cooperation  (6) 

Other  (11) 

Defense  Security  Enterprise  (17) 
Acquisition  (79) 

Installations  and  Environment  (109) 


Financial  Management  (179) 


Logistics  and  Materiei  Readiness  (344) 
Human  Resources  Management  (436) 


Source:  GAO  analysis  of  U.S.  Department  of  Defense  data.  |  GAO-15-627 


DOD  currently  bears  responsibility,  in  whole  or  in  part,  for  about  half  (17 
of  32)  of  the  programs  across  the  federal  government  that  we  have 
designated  as  high  risk.^  Seven  of  these  areas  are  specific  to  the 
department,®  and  10  other  high-risk  areas  are  shared  with  other  federal 
agencies.®  Collectively,  these  high-risk  areas  in  major  business 


^GAO-1 5-290. 

®These  seven  high-risk  areas  are  DOD’s  approach  to  business  transformation,  business 
systems  modernization,  contract  management,  financial  management,  supply  chain 
management,  support  infrastructure  management,  and  weapon  systems  acquisition. 

®The  10  governmentwide  high-risk  areas  are  climate  change  risks,  federal  disability 
programs,  technologies  critical  to  U.S.  national  security  interests,  information  systems  and 
critical  infrastructure,  information  sharing  for  homeland  security,  human  capital,  weather 
satellite  data.  Veterans  Affairs  Health  Care,  IT  acquisition  and  operations,  and  real 
property. 
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operations  are  linked  to  the  department’s  ability  to  perform  its  overall 
mission  and  affect  the  readiness  and  capabilities  of  U.S.  military  forces. 
As  such,  DOD’s  business  systems  modernization  is  one  of  the 
department’s  specific  high-risk  areas  and  is  essential  for  addressing 
many  of  the  department’s  other  high-risk  areas.  For  example,  modernized 
business  systems  are  integral  to  the  department’s  efforts  to  address  its 
financial,  supply  chain,  and  information  security  management  high-risk 
areas. 


The  NDAAfor  Fiscal  Year 
2005  Requires  DOD  to 
Improve  Its  Management 
of  Defense  Business 
Systems 


Congress  included  provisions  in  the  fiscal  year  2005  NDAA,  as  amended, 
that  are  aimed  at  ensuring  DOD’s  development  of  a  well-defined  business 
enterprise  architecture  and  associated  enterprise  transition  plan,  as  well 
as  the  establishment  and  implementation  of  effective  investment 
management  structures  and  processes. The  act  requires  DOD  to, 
among  other  things. 


•  establish  an  investment  approval  and  accountability  structure  along 
with  an  investment  review  process; 

•  not  obligate  funds  for  a  defense  business  system  program  with  a  total 
cost  in  excess  of  $1  million  over  the  period  of  the  current  future-years 
defense  program  unless  the  approval  authority  certifies  that  the 
business  system  program  meets  specified  conditions,  including 
complying  with  the  business  enterprise  architecture  and  having 
appropriate  business  process  reengineering  conducted; 

•  develop  a  business  enterprise  architecture  that  covers  all  defense 
business  systems;  and 

•  develop  an  enterprise  transition  plan  for  implementing  the 
architecture,  and  identify  systems  information  in  DOD’s  annual  budget 
submissions. 


The  fiscal  year  2005  NDAA  also  requires  that  the  Secretary  of  Defense 
submit  an  annual  report  to  the  congressional  defense  committees  on  the 
department’s  compliance  with  these  provisions.  DOD  submitted  its  most 


^“section  901  of  the  NDAA  for  fiscal  year  201 5  converts  the  position  of  the  Deputy  Chief 
Management  Officer  to  the  position  of  Under  Secretary  for  Defense  for  Business 
Management  and  Information,  effective  February  1, 2017,  thereby  reassigning 
responsibility  for  the  defense  business  system  investment  management  process  and 
enterprise  architecture  under  10  U.S.C.  §  2222  to  this  new  position. 
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recent  annual  report  to  Congress  on  April  6,  2015,  describing  steps  taken, 
underway,  and  planned  to  address  the  act’s  requirements. 


DOD’s  Approach  to 
Business  Systems 
Modernization 


DOD’s  approach  to  business  systems  modernization  includes  reviewing 
systems  annually  to  ensure  that  they  comply  with  the  fiscal  year  2005 
NDAA’s  business  enterprise  architecture  and  business  process 
reengineering  requirements.^^  This  effort  includes  both  a  certification  of 
compliance  by  lower-level  department  authorities  and  an  approval  of  this 
certification  by  higher-level  department  authorities.  According  to  the  act, 
this  certification  and  approval  is  to  occur  before  systems  are  granted 
permission  to  obligate  funds  for  a  given  fiscal  year.  These  efforts  are  to 
be  guided  by  DOD’s  Chief  Management  Officer  (CMO)  and  Deputy  Chief 
Management  Officer  (DCMO).  Specifically,  the  CMO’s  responsibilities 
include  developing  and  maintaining  a  departmentwide  strategic  plan  for 
business  reform;  establishing  performance  goals  and  measures  for 
improving  and  evaluating  overall  economy,  efficiency,  and  effectiveness; 
and  monitoring  and  measuring  the  progress  of  the  department. 

The  DCMO’s  responsibilities  include  recommending  to  the  CMO 
methodologies  and  measurement  criteria  to  better  synchronize,  integrate, 
and  coordinate  the  business  operations  to  ensure  alignment  in  support  of 
their  warfighting  mission  and  developing  and  maintaining  the 
department’s  enterprise  architecture  for  its  business  mission  area.  Table 
1  describes  selected  roles  and  responsibilities  and  the  composition  of  key 
governance  entities  and  positions  related  to  business  systems 
modernization  as  they  were  documented  for  the  fiscal  year  2015  business 
system  certification  and  approval  cycle. 


^^The  business  systems  certification  and  approval  cycle  is  the  part  of  DOD’s  overall  effort 
to  transform  its  business  operations. 

^^The  fiscal  year  201 5  business  system  certification  and  approval  process  refers  to 
certifications  and  approvals  that  were  to  occur  before  systems  were  to  obligate  funds  for 
fiscal  year  201 5. 
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Table  1:  DOD  Business  Systems  Modernization  Governance  Entities’  Roles,  Responsibilities,  and  Composition 


Entity 


Roles  and  Responsibilities 


Composition 


Deputy’s  Management  Action  Provide  strategic  direction  and  plans  for  the  business  mission 
Group/Defense  Business  area. 

Systems  Management  Recommend  policies  and  procedures  required  to  integrate 

Committee  qOD  business  transformation  and  attain  cross-department, 

end-to-end  interoperability  of  business  systems  and 
processes. 

Review  defense  business  system  portfolios. 

Serve  as  approval  authority  for  business  system  investments 
greater  than  $1  million. 

Establish  policies  and  approve  the  business  mission  area 
strategic  plan,  the  enterprise  transition  plan  for  implementation 
of  business  systems  modernization,  and  the  business 
enterprise  architecture. 


Meets  at  the  discretion  of  the  Deputy 
Secretary  of  Defense  and  is  co¬ 
chaired  by  the  Vice  Chairman  of  the 
Joint  Chiefs  of  Staff.  Includes  senior 
leadership  in  the  Office  of  the 
Secretary  of  Defense,  as  appropriate, 
such  as  the  Deputy  Chief 
Management  Officer  (DCMO)  and  the 
DOD  Chief  Information  Officer  (CIO). 
Also  includes  the  military  department 
Chief  Management  Officers  (CMO), 
the  heads  of  select  defense  agencies, 
and  participation  by  other  senior 
management,  including  from  the  Joint 
Chiefs  of  Staff  and  the  U.S. 
Transportation  Command. 


Defense  Business  Council 
/Investment  Review  Board 


Oversee  the  DOD  investment  management  process  and 
conduct  portfolio  analysis  in  support  of  the  review  and 
certification  of  covered  defense  business  system  programs.*’ 
Review  functional  strategies  developed  by  the  principal  staff 
assistants  and  assess  component  organizational  execution 
plans. 

Recommend  funds  certification  to  the  Deputy’s  Management 
Action  Group/Defense  Business  Systems  Management 
Committee. 

Prioritize  and  approve  changes  for  inclusion  in  the  business 
enterprise  architecture. 

Support  the  development  and  implementation  of  the 
department’s  end-to-end  framework. 

Validate  requirements  for  defense  business  capabilities. 

Ensure  that  defense  business  system  investments  are  aligned 
to  the  lines  of  business  for  the  department. 

Support  measurable  improvements  to  the  department’s 
business  objectives  and  generates  a  measurable  return  on 
investment. 


Co-chaired  by  the  DOD  DCMO  and 
the  DOD  CIO 
Membership  includes: 
the  Under  Secretaries  of  Defense 
(Acquisition,  Technology,  and 
Logistics;  Policy;  Personnel  and 
Readiness;  Intelligence), 

Under  Secretary  of  Defense 
(Comptroller)/Chief  Financial  Officer, 

Director  of  Cost  Assessment  and 
Program  Evaluation, 

Joint  Staff, 

DCMOs  for  the  Army,  Navy  and  Air 
Force,  CIOs  for  the  Army,  Navy  and 
Air  Force. 


Principal  Staff  Assistants  Senior  advisors  to  the  Secretary  of  Defense  that  assist  in  Under  Secretaries  of  Defense  for 

policy  development,  planning,  resource  management,  fiscal,  defined  functional  areas  (e.g., 
and  program  evaluation  responsibilities.  Comptroller;  Acquisition,  Technology, 


Develop  functional  strategies  that  are  to  describe  business 
functions,  outcomes,  measures,  and  standards  for  their 
respective  business  areas.'’ 


and  Logistics;  Intelligence;  Policy;  and 
Personnel  and  Readiness;  and  the 
DOD  CIO). 


Responsible  and  accountable  for  the  content  of  their  portions 
of  the  business  enterprise  architecture. 
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Entity 

Roles  and  Responsibilities 

Composition 

Precertification  Authority 

Ensure  component-level  investment  review  processes 
integrate  with  the  investment  management  system. 

Identify  those  component  systems  that  require  Investment 
Review  Board  certification  and  prepare,  review,  approve, 
validate,  and  transfer  investment  documentation  as  required. 
Assess  and  precertify  business  process  reengineering  and 
architecture  compliance  for  component  systems  submitted  for 
certification  and  annual  review. 

CMC  from  the  Air  Force,  Army,  Navy, 
the  director  or  the  equivalent  from  the 
defense  agencies,  and  those 
designated  by  the  DCD  DCMC  as 
appropriate  for  programs  that  support 
the  business  processes  of  more  than 
one  military  department  or  defense 
agency. 

Office  of  the  DCMO 

Develop  and  approve  business  systems  funds  certification 
process. 

Co-chair  Defense  Business  Council  /Investment  Review  Board 
with  DCD  CIC. 

Approve  business  system  certifications  (If  delegated  by  the 
Deputy  Secretary  of  Defense). 

Establish  4th  Estate  Working  Group‘d  to  support  CMC. 
Document  the  business  enterprise  architecture. 

Report  certification  results  to  Congress. 

Review  and  approve  Business  Plans  and  Business  Strategies 
(functional  strategies). 

Assess  and,  if  necessary,  optimize  business  processes  and 
practices  as  directed  by  the  DCD  CMC. 

Issue  business  process  reengineering  guidance  and  ensure 
Precertification  Authority  compliance. 

Composed  of  four  directorates: 
(Administration;  Cversight  and 
Compliance;  Management,  Policy  and 
Analysis;  and  Planning,  Performance, 
and  Assessment). 

Source:  GAO  analysis  of  DOD  information.  |  GAO-15-627. 

®The  National  Defense  Authorization  Act  for  Fiscal  Year  2015,  which  was  signed  into  law  in 
December  2014,  removed  the  requirement  for  DOD  to  maintain  the  Defense  Business  Systems 
Management  Committee.  However,  this  table  reflects  roles  and  responsibilities  as  they  applied  to  the 
fiscal  year  2015  certification  and  approval  cycle,  which  was  completed  before  this  act  was  signed  into 
law. 

'’a  covered  defense  business  system  is  any  defense  business  system  program  that  is  expected  to 
have  a  total  cost  in  excess  of  $1  million  over  the  period  of  the  current  future-years  defense  program, 
which  is  the  department’s  financial  plan  for  a  5-year  period. 

'^DOD  has  nine  functional  areas:  financial  management,  acquisition,  defense  security  enterprise, 
logistics  and  materiel  readiness,  installations  and  environment,  human  resources  management, 
security  cooperation,  enterprise  IT  infrastructure,  and  other. 

‘^The  4th  Estate  Working  Group  is  to  provide  cross-functional  review,  guidance,  and  leadership  to 
manage  and  discuss  issues  associated  with  the  other  defense  organizations  business  transformation 
efforts  and  is  also  to  provide  business  mission  area  oversight. 


Within  the  military  departments,  the  entities  described  in  table  1  are 
supported  by  portfolio  managers  who  oversee  groups  of  business  system 
investments  within  specific  functional  areas.  For  example,  the 


^^DOD  identified  a  total  of  24  business  system  portfolios  that  include  one  or  more  defense 
business  system. 
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Department  of  the  Navy’s  financial  management  portfolio  manager  is 
responsible  for  overseeing  the  Navy’s  portfolio  of  financial  management 
systems. 


Overview  of  DOD’s 
Integrated  Business 
Framework 


In  order  to  manage  and  oversee  the  department’s  business  operations 
and  approximately  1,180  covered  defense  business  systems,  the  Office 
of  the  DCMO  developed  the  Integrated  Business  Framework.  According 
to  officials  from  the  office,  this  framework  is  used  to  align  the 
department’s  strategic  objectives — laid  out  in  the  National  Security 
Strategy,''^  Quadrennial  Defense  Review,''^  and  Strategic  Management 
Plan'^^ — with  its  defense  business  system  investments. 

Using  the  overarching  goals  of  the  Strategic  Management  Plan,  principal 
staff  assistants  developed  six  functional  strategies  that  cover  nine 
functional  areas.  These  functional  strategies  are  to  define  business 
outcomes,  priorities,  measures,  and  standards  for  a  given  functional  area 
within  DOD.  The  business  objectives  and  compliance  requirements  laid 
out  in  each  functional  strategy  are  to  be  integrated  into  the  business 
enterprise  architecture. 

The  precertification  authorities  in  the  Air  Force,  Navy,  Army,  and  other 
departmental  organizations  use  the  functional  strategies  to  guide  the 
development  of  organizational  execution  plans,  which  are  to  summarize 


14 

The  National  Security  Strategy  outWnes  the  core  national  security  interests  of  the  United 
States  and  calls  fora  range  of  actions  to  implement  the  strategy. 

^®The  Quadrennial  Defense  Review  is  the  strategic  plan  for  DOD  and  sets  forth  priority 
objectives  for  DOD  and  major  actions  to  be  taken  to  accomplish  these  objectives.  The 
most  recent  review  was  issued  in  March  2014.  This  plan  is  derived  from  the  core  interests 
listed  in  the  National  Security  Strategy. 

^®The  Strategic  Management  Plan  established  specific  management  goals  that  directly 
support  the  strategic  goals  of  the  Quadrennial  Defense  Review.  DOD  has  issued  five 
updates  to  the  Strategic  Management  Plan  since  2008,  with  the  most  recent  being  issued 
in  July  2013  and  covering  fiscal  years  2014  and  2015.  According  to  officials  from  the 
Office  of  the  DCMO,  the  department  is  in  the  process  of  transitioning  from  an  approach 
focused  on  its  Strategic  Management  Plan  to  an  approach  focused  on  a  DOD  Strategic 
Plan.  However,  the  certification  and  approval  process  for  fiscal  year  2015  was  still  guided 
by  the  existing  Strategic  Management  Plan  for  fiscal  years  2014-201 5. 

^^The  functional  areas  are  acquisition,  defense  security  enterprise,  enterprise  IT 
infrastructure,  financial  management,  human  resources  management,  installations  and 
environment,  logistics  and  materiel  readiness,  security  cooperation,  and  other. 
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each  component’s  business  strategy  for  each  functional  area.  Each  plan 
includes  a  description  of  how  the  component’s  goals  and  objectives  align 
with  those  in  the  functional  strategies  and  the  Strategic  Management 
Plan.  In  addition,  each  organizational  execution  plan  includes  a  portfolio 
of  defense  business  system  investments  organized  by  functional  area. 
The  components  submit  each  of  these  portfolios  to  the  Defense  Business 
Council  for  certification  on  an  annual  basis. 

According  to  the  department’s  2015  Congressional  Report  on  Defense 
Business  Operations,  for  the  fiscal  year  2015  certification  and  review 
cycle,  the  department  empowered  the  military  department  chief 
management  officers  to  manage  their  business  systems  portfolios  and 
conduct  portfolio  reviews.  Results  were  presented  to  the  Defense 
Business  Council  and  were  to  address  topics  such  as  major 
improvements  and  cost  reductions,  return  on  investment,  risks  and 
challenges,  deviations  from  prior  plans,  and  future  goals. 

According  to  DOD’s  investment  management  guidance,  for  the  fiscal  year 
2015  certification  and  approval  cycle,  the  Defense  Business  Council  was 
to  review  the  organizational  execution  plans  and  associated  portfolios 
based  on  four  investment  criteria — compliance,  strategic  alignment,  utility, 
and  cost — to  determine  whether  or  not  to  recommend  the  portfolio  for 
certification  of  funding.  The  Vice  Chairman  of  the  Deputy’s  Management 
Action  Group/Defense  Business  Systems  Management  Committee  was  to 
approve  certification  decisions  and  then  document  the  decision  in  an 
investment  decision  memorandum.  These  memoranda  were  to  indicate 
whether  an  individual  organizational  execution  plan  has  been  certified; 
conditionally  certified  (i.e.,  obligation  of  funds  has  been  certified  and 
approved  but  may  be  subject  to  conditions  that  restrict  the  use  of  funds,  a 
time  line  for  obligation  of  funds,  or  mandatory  changes  to  the  portfolio  of 
business  systems);  or  not  certified  (i.e.,  certification  is  not  approved  due 
to  misalignment  with  strategic  direction,  mission  needs,  or  other 
deficiencies). 


Overview  of  DOD’s 
Business  Enterprise 
Architecture 


DOD’s  business  enterprise  architecture  is  intended  to  serve  as  a  blueprint 
for  the  department’s  business  transformation  efforts.  In  particular,  the 
architecture  is  to  guide  and  constrain  implementation  of  interoperable 
defense  business  systems  by,  among  other  things,  documenting  the 
department’s  business  functions  and  activities  and  the  business  rules, 
laws,  regulations,  and  policies  associated  with  them.  According  to  DOD, 
its  architecture  is  being  developed  using  an  incremental  approach,  where 
each  new  version  of  the  architecture  addresses  business  mission  area 
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gaps  or  weaknesses  based  on  priorities  identified  by  the  department.  The 
department’s  business  enterprise  architecture  focuses  on  documenting 
information  associated  with  its  end-to-end  business  process  areas  (e.g., 
hire-to-retire  and  procure-to-pay).  These  end-to-end  business  process 
areas  may  occur  across  the  department’s  nine  functional  areas.  For 
example,  hire-to-retire  occurs  within  the  human  resources  management 
functional  area,  while  the  cost  management  business  process  area 
occurs  across  the  acquisition,  financial  management,  human  resources 
management,  installations  and  environment,  and  logistics  and  materiel 
readiness  functional  areas. 

According  to  DOD  officials,  the  current  approach  to  developing  the 
business  enterprise  architecture  is  both  a  “top  down”  and  “bottom-up” 
approach.  Specifically,  the  architecture  focuses  on  developing  content  to 
support  investment  management  and  strategic  decision  making  and 
oversight  (top  down)  while  also  responding  to  department  needs 
associated  with  supporting  system  implementation,  system  integration, 
and  software  development  (bottom  up).  Consistent  with  DOD’s  tiered 
approach  to  business  systems  management,  the  department’s  approach 
to  developing  its  business  enterprise  architecture  involves  the 
development  of  a  federated  enterprise  architecture,  where  member 
architectures  (e.g..  Air  Force,  Army,  and  Navy)  conform  to  an  overarching 
corporate  or  parent  architecture  and  use  a  common  vocabulary.  This 
approach  is  to  provide  governance  across  all  business  systems, 
functions,  and  activities  within  the  department  and  improve  visibility 
across  the  respective  efforts. 


Overview  of  DOD’s 
Business  Process 
Reengineering  Efforts 


DOD  defines  business  process  reengineering  as  a  logical  methodology 
for  assessing  process  weaknesses,  identifying  gaps,  and  implementing 
opportunities  to  streamline  and  improve  the  processes  to  create  a  solid 
foundation  for  success  in  changes  to  the  full  spectrum  of  operations. 
DOD’s  reengineering  efforts  are  intended  to  help  the  department 
rationalize  its  covered  business  system  portfolio,  improve  its  use  of 
performance  management,  control  scope  changes,  and  reduce  the  cost 
of  fielding  business  capability.  According  to  DOD  officials,  the  department 
has  taken  a  holistic  approach  to  business  process  reengineering,  which 
includes  a  portfolio  and  end-to-end  perspective.  It  has  also  issued 
business  process  reengineering  guidance  that  calls  for  alignment  of 
defense  business  systems  within  the  Organizational  Execution  Plan  to  its 
functional  strategy’s  strategic  goals. 
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An  important  component  of  the  department’s  business  process 
reengineering  efforts  is  the  problem  statement  development  and  review 
process.  A  problem  statement  is  developed  when  a  defense  business 
system  is  seeking  certification  for  a  development  or  modernization  effort. 
The  statement  is  to  include,  among  other  things,  a  description  of  the 
problem  that  the  system  intends  to  address  and  a  discussion  of  the  costs, 
benefits,  and  risks  of  various  alternatives  that  were  considered.  As  part  of 
the  annual  certification  and  approval  process,  problem  statements  are  to 
be  reviewed  to  support  that  appropriate  business  process  reengineering 
has  been  conducted  on  investments  seeking  certification. 


DOD  Has 
Made  Progress 
Strengthening  Its 
Management  of 
Business  Systems 
but  Needs 
to  Continue 
Implementing 
Recommended 
Improvements 


The  department  has  implemented  5  of  the  16  recommendations  that  GAO 
has  made  since  June  201 1  to  address  each  of  the  overarching  provisions 
for  improving  business  systems  management  in  the  fiscal  year  2005 
NDAA.  The  fiscal  year  2005  NDAA,  as  amended,  includes  provisions 
associated  with  developing  a  business  enterprise  architecture  and 
enterprise  transition  plan,  improving  the  department’s  investment 
management  structures  and  processes,  improving  its  efforts  to  certify 
defense  business  systems,  and  mandated  budgetary  reporting.  Since 
201 1 ,  we  have  issued  four  reports  in  response  to  the  act’s  requirement 
that  we  assess  the  actions  taken  by  the  department  to  comply  with  the 
act’s  provisions.''®  In  those  reports,  we  have  made  recommendations  to 
address  each  of  the  act’s  overarching  provisions  for  improving  business 
systems  management.  Table  2  identifies  the  recommendations  we  have 
made  since  201 1  associated  with  the  fiscal  year  2005  NDAA. 


^®See  GAO-14-486,  GAO-13-557,  GAO-12-685,  GAO-11-684. 
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Table  2:  Fiscal  Year  2005  NDAA  Provisions  and  Associated  GAO  Recommendations 


NDAA  provision 

Associated  Recommendations 

Business  Enterprise  Architecture 

•  Complete  the  transfer  of  functions  of  the  Business  Transformation  Agency  and  the  Office 
of  the  Assistant  Secretary  of  Defense  for  Networks  and  Information 

Integration/Department  of  Defense  Chief  Information  Officer  (CIO)  and  specify  when  and 
where  these  functions  will  be  transferred.  (GAO-11-684) 

•  Establish  a  policy  that  clarifies  the  roles,  responsibilities,  and  relationships  among  the 

Chief  Management  Officer,  Deputy  Chief  Management  Officer  (DCMO),  DOD  and  military 
department  CIOs,  Principal  Staff  Assistants,  military  department  Chief  Management 
Officers,  and  the  heads  of  the  military  departments  and  defense  agencies,  associated 
with  the  development  of  a  federated  business  enterprise  architecture.  (GAO-12-685) 

•  Define  by  when  and  how  the  department  plans  to  develop  an  architecture  that  would 
extend  to  all  defense  components.  (GAO-13-557) 

Enterprise  Transition  Plan 

•  Define  by  when  and  how  the  enterprise  transition  plan  will  include  key  elements  such  as 
milestones,  performance  measures,  and  funding  plans;  dates  for  terminating  legacy 
systems;  a  list  of  business  systems  that  will  be  part  of  the  target  environment  and  a 
strategy  for  modifying  them,  as  appropriate;  and  information  about  sequencing  systems. 
(GAO-13-557) 

Investment  Management 

•  Ensure  that  the  functional  strategies  include  all  of  the  critical  elements  identified  in  DOD 
investment  management  guidance.  (GAO-13-557) 

•  Select  and  control  its  mix  of  investments  in  a  manner  that  best  supports  mission  needs  by 
documenting  a  process  for  evaluating  portfolio  performance;  ensuring  that  portfolio 
assessments  are  conducted  in  key  areas;  and  ensuring  that  investment  management 
documentation  includes  critical  information  for  conducting  all  assessments.  (GAO-13-557) 

•  Define  criteria  for  reviewing  defense  business  systems  at  an  appropriate  level  in  the 
department  based  on  factors  such  as  complexity,  scope,  cost,  and  risk,  in  support  of  the 
certification  and  approval  process.  (GAO-14-486) 

•  Develop  guidance  requiring  military  departments  and  other  defense  organizations  to  use 
existing  business  enterprise  architecture  content  to  more  proactively  identify  potential 
duplication  and  overlap.  (GAO-14-486) 

Investment  Certification 
and  Approval 

•  Include  in  DOD’s  annual  report  to  Congress  on  compliance  with  1 0  U.S.C.  §  2222  the 
results  of  the  department’s  business  process  reengineering  efforts.  (GAO-12-685) 

•  Implement  and  use  the  business  enterprise  architecture  and  business  process 
reengineering  compliance  assessments  more  effectively  to  support  organizational 
transformation  efforts  by  disclosing  relevant  information  about  known  weaknesses; 
establishing  milestones  for  completing  selected  validations  of  business  enterprise 
architecture  compliance  assertions;  and  ensuring  that  appropriate  business  process 
reengineering  assertions  have  been  completed  on  all  investments  submitted  for  the  fiscal 
year  2014  certification  reviews  prior  to  the  certification  of  funds.  (GAO-13-557) 

•  Ensure  that  complete  documentation,  such  as  root  cause  analyses,  assessments  of 
existing  interfaces  for  reuse  opportunities,  and  performance  metrics  related  to  the 
reengineering  efforts,  is  provided  as  part  of  the  fiscal  year  2014  certification  and  approval 
process  for  selected  systems.  (GAO-13-557) 

•  Determine  if  funds  were  properly  obligated  under  1 0  U.S.C.  2222(a)-(b)  for  systems  for 
which  appropriate  business  process  reengineering  assertions  were  not  completed. 
(GAO-13-557) 

•  Define  by  when  and  how  the  department  plans  to  align  its  business  system  certification 
and  approval  process  with  its  Planning,  Programming,  Budgeting,  and  Execution  process. 
(GAO- 14-486) 
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NDAA  provision 

Associated  Recommendations 

Mandated  Budgetary  Reporting 

•  Establish  a  deadline  for  integrating  the  repositories  and  validate  the  completeness  and 
reliability  of  information.  (GAO-12-685) 

Other  (Human  Capital) 

•  Include  in  DOD’s  annual  report  to  Congress  on  compliance  with  1 0  U.S.C.  §  2222  an 
update  on  the  office  of  the  DCMO’s  progress  toward  filling  staff  positions  and  the  impact 
of  any  unfilled  positions  on  the  ability  of  the  office  to  conduct  its  work.  (GAO-12-685) 

•  Develop  a  skills  inventory,  needs  assessment,  gap  analysis,  and  plan  to  address 
identified  gaps  as  part  of  a  strategic  approach  to  human  capital  planning  for  the  Office  of 
the  Deputy  Chief  Management  Officer.  (GAO-13-557) 

Source:  GAO  analysis.  |  GAO-15-627. 


Table  3  presents  a  summary  of  the  current  status  of  these 
recommendations.  Appendix  II  provides  additional  information  about  the 
status  of  each  recommendation. 


Table  3:  Status  of  Recommendations  Associated  with  the  Fiscal  Year  2005  NDAA 

NDAA  provision 

Implemented 

Partially 

implemented 

Total 

Business  Enterprise  Architecture 

1 

2 

3 

Enterprise  Transition  Plan 

- 

1 

1 

Investment  Management 

1 

3 

4 

Investment  Certification  and  Approval 

2 

3 

5 

Mandated  Budgetary  Reporting 

1 

- 

1 

Other  (Human  Capital) 

- 

2 

2 

Total 

5 

11 

16 

Source:  GAO  Analysis  of  DOD  Documentation.  |  GAO-15-627. 


As  of  April  2015,  the  department  had  implemented  5  of  the  16 
recommendations  that  we  have  made  since  June  201 1 .  For  example,  the 
department  has  implemented  the  recommendation  to  improve  its 
reporting  of  business  system  data  in  its  annual  budget  request.  In 
particular,  the  department  has  established  common  elements  in  its  three 
primary  repositories  used  for  tracking  information  about  business 
systems,  which  allows  information  about  individual  business  systems  to 
be  matched  across  systems.  In  addition,  the  Office  of  the  CIO 
demonstrated  that  it  conducts  periodic  data  quality  assessments.  As  a 
result,  the  department  is  better  positioned  to  report  more  reliable 
information  in  its  annual  budget  request  and  to  maintain  more  accurate 
information  about  business  systems  to  support  its  efforts  to  manage 
them. 
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In  addition,  the  department  has  improved  the  alignment  of  its  Planning, 
Programming,  Budgeting,  and  Execution  process  with  its  business 
systems  certification  and  approval  process.  For  example,  according  to  the 
department’s  February  2015  certification  and  approval  guidance. 
Organization  Execution  Plans  are  to  include  information  about 
certification  requests  for  the  upcoming  fiscal  year  as  well  as  over  the 
course  of  the  Future  Years  Defense  Program.  As  a  result,  the 
department’s  business  system  certification  and  approval  process  can 
support  better  informed  decisions  about  system  certifications  and  inform 
recommendations  on  the  resources  provided  to  defense  business 
systems  as  part  of  the  Planning,  Programming,  Budgeting,  and  Execution 
process. 

The  department  has  partially  implemented  the  remaining  11 
recommendations.  For  example,  the  department’s  February  2015 
investment  management  guidance,  which  describes  DOD’s  business 
system  certification  and  approval  process,  identifies  four  criteria  and 
specifies  the  associated  assessments  that  are  to  be  conducted  when 
reviewing  and  evaluating  component-level  organizational  execution  plans 
in  order  to  make  a  portfolio-based  investment  decision.  The  guidance 
also  states  that  return  on  investment  should  be  considered  when 
evaluating  program  cost.  However,  it  does  not  call  for  the  use  of  actual- 
versus-expected  performance  data  and  predetermined  thresholds. 

Further,  the  Office  of  the  DCMO  has  developed  a  draft  resource 
allocation  plan  for  each  of  its  directorates  and  their  respective  divisions. 
This  draft  plan  includes  staffing  profiles  that  describe  each  division’s 
needed  staff  competencies  and  qualifications.  However,  the  Office  of  the 
DCMO  did  not  demonstrate  that  it  has  addressed  other  important  aspects 
of  strategic  human  capital  planning.  For  example,  the  office  did  not 
demonstrate  that  it  has  developed  a  skills  inventory,  needs  assessment, 
gap  analysis,  and  plan  to  address  identified  gaps,  as  called  for  by  our 
recommendation.  Appendix  II  provides  additional  information  about  the 
recommendations  that  DOD  has  fully  and  partially  implemented. 

Implementing  the  remaining  1 1  recommendations  will  improve  DOD’s 
modernization  management  controls  and  help  fulfill  the  department’s 
execution  of  the  requirements  of  the  act. 
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DOD  Needs  to 
Improve  the 
Usefulness  and 
Effectiveness  of  Its 
Business  Enterprise 
Architecture  and 
Business  Process 
Reengineering 
Initiatives 


DOD’s  business  enterprise  architecture  and  process  reengineering  efforts 
are  not  fully  achieving  the  intended  outcomes  described  in  statute.  More 
specifically,  with  respect  to  the  architecture,  portfolio  managers 
(managers)  we  surveyed  reported  that  it  was  generally  not  effective  in 
achieving  its  intended  outcomes  and  that  its  usefulness  in  achieving 
benefits,  such  as  reducing  the  number  of  applications,  was  limited.  With 
respect  to  process  reengineering,  managers  reported  these  efforts  were 
moderately  effective  at  streamlining  business  processes,  but  less  so  in 
limiting  the  need  to  tailor  commercial  off-the-shelf  systems. 

Portfolio  managers  cited  a  number  of  challenges  impeding  the  usefulness 
and  effectiveness  of  these  two  initiatives,  such  as  the  availability  of 
training,  lack  of  skilled  staff,  parochialism,  and  cultural  resistance  to 
change.  DOD  has  various  improvement  efforts  underway  to  address 
some  of  these  challenges;  however,  additional  work  is  needed  and  the 
managers  provided  some  suggestions  for  closing  the  gap.  More  fully 
addressing  the  cited  challenges  would  help  increase  the  utility  and 
effectiveness  of  these  initiatives  in  driving  greater  operational  efficiencies 
and  savings.  Appendix  I  provides  additional  details  about  our  survey 
methodology. 


The  NDAAfor  Fiscal  Year 
2005  Defines  Intended 
Outcomes  for  the 
Business  Enterprise 
Architecture 


The  fiscal  year  2005  NDAA,  as  amended,  requires  DOD  to  develop  a 
business  enterprise  architecture  that  covers  all  defense  business  systems 
and  will  be  used  as  a  guide  for  these  systems.  According  to  the  act,  the 
architecture  is  intended  to  help  achieve  the  following  outcomes: 

•  Enable  DOD  to  comply  with  all  applicable  laws,  including  federal 
accounting,  financial  management,  and  reporting  requirements. 

•  Guide,  permit,  and  constrain  the  implementation  of  interoperable 
defense  business  systems. 

•  Enable  DOD  to  routinely  produce  timely,  accurate,  and  reliable 
business  and  financial  information  for  management  purposes. 

•  Facilitate  the  integration  of  budget,  accounting,  and  program 
information  and  systems. 

•  Provide  for  the  systematic  measurement  of  performance,  including  the 
ability  to  produce  timely,  relevant,  and  reliable  cost  information. 


The  act  also  specifies  that  the  department  is  not  to  obligate  funds  for 
defense  business  system  programs  that  have  a  total  cost  in  excess  of  $1 
million  unless  the  system’s  approval  authority  certifies  that  the  program 


Page  16 


GAO-15-627  DOD  Business  Systems  Modernization 


complies  with  the  business  enterprise  architecture  and  the  certification  is 
subsequently  approved  by  the  department’s  Investment  Review  Board. 

Achieving  the  act’s  intended  outcomes  would  contribute  to  the 
department’s  ability  to  use  the  architecture  to  realize  important  benefits 
that  we  and  others  have  previously  identified,  such  as  cost  savings  or 
avoidance.''®  For  example,  if  the  architecture  effectively  guides,  permits, 
and  constrains  the  implementation  of  interoperable  systems,  that  would 
contribute  to  increased  information  sharing  and  improved  system 
interoperability.  As  another  example,  using  the  architecture  to  produce 
timely  and  reliable  business  and  financial  information  would  contribute  to 
improving  management  decisions  associated  with  enhanced  productivity 
and  improved  business  and  IT  alignment,  among  other  things. 


The  Business  Enterprise 
Architecture  Has 
Produced  Limited  Value, 
According  to  Portfolio 
Managers 


The  majority  of  DOD  portfolio  managers  we  surveyed  reported  that  the 
business  enterprise  architecture  has  not  been  effective  in  meeting  its 
intended  outcomes.  More  specifically,  half  of  the  managers  surveyed 
reported  that  the  business  enterprise  architecture  was  effective  in 
enabling  compliance  with  all  applicable  laws.  However,  fewer  than  40 
percent  reported  that  the  architecture  was  effective  in  helping  to  achieve 
the  other  outcomes  called  for  by  the  fiscal  year  2005  NDAA.  Table  4 
provides  additional  information  on  survey  responses  regarding  the  act’s 
specific  requirements. 


1 Q 

See,  for  example,  GAO,  Organizational  Transformation:  Enterprise  Architecture  Value 
Needs  to  Be  Measured  and  Reported,  GAO-12-791  (Washington,  D.C.:  Sept.  26,  2012) 
and  Organizational  Transformation:  A  Framework  for  Assessing  and  Improving  Enterprise 
Architecture  Management  (Version  2.0),  GAO-1 0-846G  (Washington,  D.C.:  Aug  5,  2010). 
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Table  4:  Effectiveness  of  the  Business  Enterprise  Architecture  in  Achieving  Fiscal  Year  2005  NDAA  Outcomes  Reported  by 
DOD  Portfolio  Managers  (by  Percent  in  Agreement) 


Expected  business  enterprise  architecture  outcome 

Effective 

Neither 
effective  nor 
ineffective 

ineffective 

Not 

appiicabie/no 
basis  to  judge 

Enables  DOD  to  comply  with  all  applicable  laws,  including 
federal  accounting,  financial  management,  and  reporting 
requirements. 

50 

25 

17 

8 

Guides  the  implementation  of  interoperable  defense 
business  systems. 

38 

29 

29 

4 

Permits  implementation  of  interoperable  defense  business 
systems. 

33 

46 

17 

4 

Enables  DOD  to  routinely  produce  timely,  accurate,  and 
reliable  business  and  financial  information  for 
management  purposes. 

25 

25 

33 

17 

Enables  DOD  to  integrate  budget,  accounting,  and 
program  information  and  systems. 

21 

17 

38 

25 

Provides  for  the  systematic  measurement  of  performance, 
including  the  ability  to  produce  timely,  relevant,  and 
reliable  cost  information. 

21 

25 

33 

21 

Constrains  the  implementation  of  interoperable  defense 
business  systems. 

13 

58 

13 

17 

Source:  GAO  analysis  of  DOD  survey  responses.  |  GAO-15-627. 

Note:  Numbers  might  not  add  to  100  percent  due  to  rounding. 


Portfolio  managers  provided  additional  details  to  further  explain  their 
survey  responses.  Their  comments  included  the  following: 

•  The  architecture  is  a  standalone  effort  that  does  not  drive 
comprehensive  portfolio  and  business  management  through  the 
various  DOD  components. 

•  The  architecture  is  overwhelming  to  review  and  is  not  integrated  with 
other  activities  that  occur  throughout  the  remainder  of  the  year. 

•  The  compliance  requirements  are  not  sufficiently  defined  to  enable 
system  interoperability. 

Portfolio  managers  also  reported  that  the  usefulness  of  DOD’s  business 
enterprise  architecture  in  achieving  various  potential  benefits  is  limited. 
For  example,  75  percent  reported  limited  achievement  of  improved 
change  management  and  74  percent  reported  limited  achievement  of 
streamlined  end-to-end  business  processes.  In  addition,  71  percent 
reported  limited  achievement  of  benefits  such  as  a  reduced  number  of 
applications,  improved  business  and  IT  alignment,  enhanced  productivity. 
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and  achieving  financial  benefits  such  as  cost  savings  or  cost  avoidance. 
Table  5  summarizes  the  portfolio  managers’  survey  responses. 

Table  5:  Benefits  Achieved  by  the  Business  Enterprise  Architecture  Reported  by  DOD  Portfolio  Managers  (by  Percent  in 
Agreement) 

Very  great,  great,  or 

Some,  little.  Not  applicable/no  basis 

Benefit 

moderate  extent 

or  no  extent 

to  judge 

Improved  change  management 

17 

75 

8 

streamlined  end-to-end  business  processes 

26 

74 

0 

Reduced  the  number  of  applications 

29 

71 

0 

Improved  business  and  IT  alignment 

29 

71 

0 

Enhanced  productivity 

25 

71 

4 

Achieved  financial  benefits,  such  as  cost 
savings  or  cost  avoidance 

21 

71 

8 

Increased  use  of  enterprise  licenses 

13 

67 

21 

Consolidation  of  data  centers 

4 

67 

29 

Increased  information  sharing 

34 

63 

4 

Improved  system  interoperability 

34 

58 

8 

Source:  GAO  analysis  of  DOD  survey  responses.  |  GAO-15-627. 

Note:  Numbers  might  not  add  to  100  percent  due  to  rounding. 


Although  managers  reported  limited  achievement  of  benefits,  two 
provided  specific  examples  of  individual  benefits  associated  with  the 
business  enterprise  architecture.  More  specifically,  one  cited  saving  $10 
million  annually  due  to  the  establishment  of  a  DOD-wide  military  housing 
system  that  has  replaced  a  number  of  individual  systems.  A  second 
reported  $11.5  million  in  architecture-related  savings  through  the 
retirement  of  48  real  property  and  financial  management  systems. 

In  addition,  officials  from  the  Office  of  the  DCMO  provided  specific 
examples  of  benefits  that  they  stated  can  be  attributed,  at  least  in  part,  to 
the  department’s  business  architecture.  For  example,  according  to  these 
officials,  two  proposed  new  defense  business  system  investments  were 
not  approved  by  DOD  due,  in  part,  to  architecture  reviews  that  revealed 
the  requested  capabilities  were  already  available  in  existing  systems. 


Numerous  Challenges  in 
Achieving  Intended  Business 
Enterprise  Architecture 
Outcomes  Identified  by 
Portfolio  Managers 


The  surveyed  DOD  portfolio  managers  reported  that  their  functional  areas 
face  many  challenges  in  achieving  the  outcomes  described  in  the  NDAA 
for  fiscal  year  2005.  The  most  frequently  cited  challenges  reported  were 
the  usability  of  the  compliance  tool  (79  percent),  frequent  changes  to  the 
architecture  (75  percent),  the  availability  of  training  (71  percent),  the 
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availability  of  skilled  staff  (71  percent),  parochialism  (67  percent),  and 
cultural  resistance  to  change  (63  percent).  Table  6  identifies  the  survey 
responses  to  achieving  the  architecture’s  intended  outcomes. 

Table  6:  Challenges  to  Achieving  the  Business  Enterprise  Architecture’s  Intended  Outcomes  Reported  by  DOD  Portfolio 
Managers  (by  Percent  in  Agreement) 

Challenge 

Very  great,  great, 
or  moderate  extent 

Slight  challenge  or  Not  applicable/no 
not  a  challenge  basis  to  judge 

Usability  of  DOD’s  business  enterprise  architecture  compliance  tool 

79 

21 

0 

Frequent  changes  to  the  business  enterprise  architecture 

75 

21 

T 

Availability  of  training 

71 

25 

T 

Availability  of  skilled  staff 

71 

25 

T 

Parochialism  (i.e.,  focusing  on  one’s  own  sub-organization  rather 
than  having  an  enterprise-wide  view.) 

67 

29 

T 

Cultural  resistance  to  change 

63 

33 

T 

Limited  alignment  between  corporate  and  component  business 
enterprise  architectures 

58 

38 

T 

Business  enterprise  architecture  compliance  review  process 

54 

46 

~0~ 

Limited  detail  in  corporate  business  enterprise  architecture 

50 

46 

T 

Limited  detail  in  business  enterprise  architecture  compliance 
guidance 

50 

46 

T 

Quality  of  business  enterprise  architecture  compliance  guidance 

50 

46 

T 

Limited  detail  in  component  (e.g.,  military  department)  business 
enterprise  architecture 

48 

43 

T 

Source:  GAO  analysis  of  DOD  survey  responses.  |  GAO-15-627. 

Note:  Numbers  might  not  add  to  100  percent  due  to  rounding. 


Deputy  Chief  Management 
Office  Has  Taken  Steps  to 
Improve  Business  Enterprise 
Architecture  Content  and 
Use,  but  Survey  Results 
Indicate  Additional  Measures 
Are  Needed 


Officials  from  the  Office  of  the  DCMO,  including  the  Lead  Architect  for  the 
business  enterprise  architecture  and  the  Chief  of  Portfolio  Management, 
described  various  efforts  underway  to  address  selected  challenges 
identified  in  our  survey  results.  With  regard  to  the  top  ranked  challenge 
(usability  of  DOD’s  architecture  compliance  tool),  the  office  has  been 
working  on  a  more  robust  replacement  tool.  As  of  April  2015,  the  office 
had  moved  architecture  content  and  associated  compliance  information 
from  its  previous  tool  into  its  Integrated  Business  Framework-Data 
Alignment  Portal.  Further,  the  department  plans  to  require  all  fiscal  year 
2016  compliance  assessments  to  be  completed  in  this  portal 
environment. 


According  to  officials  from  the  Office  of  the  DCMO,  this  change  will  help 
ensure  that  architecture-related  information  is  available  in  the  same 
place,  which  will  help  support  more  sophisticated  analysis  of  information 
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about  business  systems.  For  example,  by  combining  information  about 
the  architecture,  compliance  information,  functional  strategies,  and 
organizational  execution  plans,  the  department  could  more  easily  conduct 
analyses  that  will  help  support  portfolio  management.  According  to  these 
officials,  examples  of  such  analyses  include  the  ability  to  identify  the 
funds  certified  and  approved  for  various  business  activities  and  the  ability 
to  identify  systems  that  conduct  similar  system  functions. 

With  regard  to  the  challenge  associated  with  limited  alignment  between 
corporate  and  component  architectures,  the  officials  from  the  Office  of  the 
DCMO  stated  that  they  intend  to  develop  an  overarching  (or  federated) 
architecture  that  will  capture  content  from,  and  allow  governance  across, 
the  department  (e.g..  Army,  Navy,  and  Air  Force).  We  previously 
recommended  that  DOD  establish  a  plan  for  how  it  will  address  business 
enterprise  architecture  federation  in  2013.^° 

The  department’s  improvement  efforts  only  address  selected  reported 
challenges.  However,  portfolio  managers  offered  a  number  of 
suggestions  that  relate  to  other  identified  challenges  that  may  help  close 
gaps  in  these  efforts.  Key  suggestions  included: 

•  Improve  tools:  Four  of  24  managers  offered  suggestions  that  relate 
to  compliance  tool  usability.  For  example,  one  portfolio  manager 
stated  that  functionality  should  be  added  to  the  architecture 
compliance  tool  to  automatically  create  and  build  the  architecture 
artifacts  mentioned  in  compliance  guidance  using  the  information 
already  included  in  the  tool  for  each  system.  Another  portfolio 
manager  stated  that  there  are  no  tools  available  that  portfolio 
managers  can  use  to  analyze  their  portfolios  relative  to  the 
architecture. 

•  Provide  additional  training:  Two  managers  offered  suggestions 
associated  with  additional  training.  For  example,  one  manager 
reported  that  the  compliance  tool  is  not  user  friendly  and  little  to  no 
training  was  offered  when  programs  were  required  to  use  it  to  assert 
compliance.  As  a  result,  this  manager  added  that  more  training  should 
be  made  available  for  using  the  compliance  tool. 

•  Start  the  process  earlier  in  a  system’s  life  cycle:  One  manager 
suggested  the  architecture  be  addressed  earlier  in  the  acquisition  life 


2°GA0-1  3-557. 
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cycle,  such  as  in  the  analysis  of  alternatives  phase,  in  order  to  help 
assess  whether  existing  solutions  are  already  employed  in  other 
areas  of  the  enterprise.  If  the  architecture  compliance  process 
uncovers  potential  duplication  or  overlap,  it  might  be  easier  to  stop 
development  of  a  duplicative  system  earlier  in  its  life  cycle  rather  than 
waiting  until  a  business  process  is  more  reliant  on  a  planned  system 
that  is  closer  to  becoming  operational. 

•  Establish  priorities:  One  portfolio  manager  suggested  that  the 
department  develop  departmental  business  improvement  and 
integration  priorities^''  and  develop  clearly  understandable  and 
verifiable  compliance  standards  that  will  guide  and  constrain  systems 
development  to  help  achieve  those  priorities. 

•  Improve  guidance:  Two  managers  suggested  that  the  department 
improve  its  guidance  to  clarify  the  documentation  that  systems 
developed  prior  to  the  existence  of  the  business  enterprise 
architecture  are  required  to  prepare  to  address  the  business 
enterprise  architecture  compliance  requirement. 

•  Improve  content:  Seven  managers  offered  suggestions  associated 
with  improving  content.  For  example,  one  manager  stated  that  the 
business  enterprise  architecture  is  large  and  cumbersome  and 
incomplete  in  many  areas. 

Addressing  the  challenges  cited  by  the  portfolio  managers  could  help 
increase  the  utility  and  effectiveness  of  the  department’s  business 
enterprise  architecture  in  driving  greater  operational  efficiencies  and  cost 
savings. 


The  NDAAfor  Fiscal  Year 
2005  Defines  Intended 
Outcomes  for  Business 
Process  Reengineering 


The  fiscal  year  2005  NDAA,  as  amended,  establishes  expected  outcomes 
for  the  department’s  business  process  reengineering  efforts.  The  act 
states  that  funds  for  covered  business  system  programs  cannot  be 
certified  and  approved  unless  each  program’s  pre-certification  authority 
has  determined  that,  among  other  things,  appropriate  business  process 
reengineering  efforts  have  been  undertaken  to  ensure  that  the  business 
process  supported  by  the  program  is,  or  will  be,  as  streamlined  and 


have  previously  recommended  that  DOD  document  and  communicate  business 
transformation  priorities.  See  GAO,  Defense  Business  Transformation:  DOD  Has  Taken 
Some  Steps  to  Address  Weaknesses,  but  Additionai  Actions  Are  Needed,  GAO-1 5-213 
(Washington,  D.C.:  Feb.  11,2015). 
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efficient  as  practicable  and  the  need  to  tailor  commercial  off-the-shelf 
systems  to  (a)  meet  unique  requirements,  (b)  incorporate  unique 
requirements,  or  (c)  incorporate  unique  interfaces  has  been  eliminated  or 
reduced  to  the  maximum  extent  practicable.  As  we  have  previously 
reported,  modifications  to  commercial  off-the-shelf  systems  should  be 
avoided  to  the  extent  practicable  as  they  can  be  costly  to  implement.^^ 

Achieving  the  intended  outcomes  of  the  fiscal  year  2005  NDAA  would 
increase  the  department’s  ability  to  realize  key  benefits  to  business 
systems  modernization.  For  example,  reengineering  business  processes 
to  be  as  streamlined  as  possible  can  result  in  increased  efficiencies,  a 
reduced  number  of  interfaces,  and  decreased  program  costs. 


DOD’s  Reengineering 
Efforts  Have  Moderately 
Helped  to  Streamline 
Business  Processes  and 
Realize  Some  Efficiencies, 
but  More  Work  Is  Needed 
to  Fully  Achieve  Intended 
Outcomes,  According  to 
Portfolio  Managers 


The  department’s  business  process  reengineering  efforts  have  had  mixed 
success  in  achieving  their  intended  outcomes.  Specifically,  63  percent  of 
the  portfolio  managers  we  surveyed  reported  that  the  efforts  were 
effective  in  helping  to  ensure  that  the  business  processes  supported  by 
the  defense  business  systems  they  manage  are  (or  will  be)  streamlined 
and  efficient  as  practicable.  As  an  example,  one  manager  reported  this 
effort  highlighted  the  strengths  and  weaknesses  of  the  systems  within 
their  specific  portfolio.  Another  reported  that  their  portfolio  has  been 
reduced  from  147  systems  to  13  due,  in  part,  to  the  business  process 
reengineering  efforts. 

However,  the  general  consensus  among  surveyed  portfolio  managers 
was  that  the  department’s  efforts  were  less  effective  in  helping  to  limit 
tailoring  of  commercial  off-the-shelf  systems.  Only  29  percent  reported 
that  DOD’s  business  process  reengineering  efforts  were  effective  in 
eliminating  or  reducing  the  need  to  tailor  commercial  off-the-shelf 
systems.  Tailoring  might  be  required,  for  example,  because  existing 
policy  and  guidance  might  limit  a  system’s  ability  to  conform  to  a  specific 
approach  for  executing  a  business  process  that  is  already  built  into  an 
individual  commercial  off-the-shelf  system.  Another  reason  given  was  that 
managers  have  limited  knowledge  about  the  commercial  off-the-shelf 
products  that  are  available  via  established  enterprise  licenses  and  this 


^^GAO,  Cost  Estimating  and  Assessment  Guide,  GAO-09-03SP  (Washington,  D.C.: 
March  2009). 
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limited  knowledge  makes  it  difficult  to  conduct  effective  business  process 
reengineering. 

Table  7  provides  additional  information  on  portfolio  managers’  responses 
regarding  the  effectiveness  of  DOD’s  business  process  reengineering 
efforts. 


Table  7:  Effectiveness  of  Business  Process  Reengineering  in  Achieving  Fiscal  Year  2005  NDAA  Requirements  Reported  by 
Portfolio  Managers  (by  Percent  in  Agreement) 


Requirement 

Effective 

Neither 
effective  nor 
ineffective 

Ineffective 

Not 

applicable/no 
basis  to  judge 

Business  process  supported  by  defense  business  systems  are 
(or  will  be)  as  streamlined  and  efficient  as  practicable. 

63 

17 

17 

4 

The  need  to  tailor  commercial  off-the-shelf  systems  to  meet 
unique  requirements  has  been  eliminated  or  reduced  to  the 
maximum  extent  practicable 

29 

25 

38 

8 

The  need  to  tailor  commercial  off-the-shelf  systems  to 
incorporate  unique  requirements  has  been  eliminated  or 
reduced  to  the  maximum  extent  practicable. 

29 

25 

38 

8 

The  need  to  tailor  commercial  off-the-shelf  systems  to 
incorporate  unique  interfaces  has  been  eliminated  or  reduced 
to  the  maximum  extent  practicable. 

29 

21 

42 

8 

Source:  GAO  analysis  of  DOD  survey  responses.  |  GAO-15-627. 

Note:  Numbers  might  not  add  to  100  percent  due  to  rounding. 


Portfolio  managers  reported  that  business  process  reengineering  has 
been  useful  in  helping  to  achieve  selected  benefits.  In  particular,  70 
percent  reported  that  efforts  have  resulted  in  streamlined  business 
processes.  Sixty-seven  percent  reported  that  efforts  have  resulted  in 
improved  documentation  of  business  needs,  which  is  consistent  with 
DOD’s  focus  on  developing  problem  statements  for  new  capabilities. 

Such  problem  statements  reflect  analysis  of  a  perceived  business 
problem,  capability  gap,  or  opportunity.  According  to  officials  from  the 
Office  of  the  DCMO,  they  help  ensure  that  programs  are  aligned  with 
DOD’s  strategic  needs,  and  also  assist  the  department’s  efforts  in 
identifying  redundancies  and  duplication.  However,  only  29  percent  of  the 
portfolio  managers  surveyed  reported  that  efforts  to  reduce  program  costs 
have  been  effective.  Table  8  summarizes  the  portfolio  managers’  survey 
responses. 
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Table  8:  Extent  to  Which  Business  Process  Reengineering  Benefits  Were  Reported  by  DOD  Portfolio  Managers  (by  Percent  in 
Agreement) 

Benefit 

Very  great,  great,  or 
moderate  extent 

Some,  little, 
or  none 

Not  applicable/ 
no  basis  to  judge 

Streamlined  business  processes 

70 

30 

0 

Improved  documentation  of  business  need 

67 

29 

4 

Increased  efficiencies 

54 

42 

4 

Reduced  interfaces 

46 

50 

4 

Decreased  program  costs 

29 

58 

13 

Source:  GAO  analysis  of  DOD  survey  responses.  |  GAO-15-627. 

Note:  Numbers  might  not  add  to  100  percent  due  to  rounding. 

Numerous  Business  The  surveyed  DOD  portfolio  managers  identified  a  range  of  challenges  to 

Process  Reengineering  achieving  the  business  process  reengineering  outcomes  described  in 

Challenges  Identified  fiscal  year  2005  NDAA.  In  particular,  cultural  resistance  to  change 

was  the  most  frequently  cited  challenge  (71  percent),  followed  by 
parochialism  (i.e.,  focusing  on  one’s  own  sub-organization  rather  than 
having  an  enterprise-wide  view.),  availability  of  skilled  staff,  and 
availability  of  training  (all  at  67  percent).  The  quality  of  business  process 
reengineering  compliance  guidance,  the  compliance  review  process,  and 
the  timing  of  the  reengineering  relative  to  system  development  work  were 
also  reported  as  important  challenges  (all  at  63  percent). 

Table  9  summarizes  survey  responses  to  questions  about  the  challenges 
to  business  process  reengineering. 
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Table  9:  Challenges  to  Achieving  Intended  Business  Process  Reengineering  Outcomes  Reported  by  DOD  Portfolio  Managers 
(by  Percent  in  Agreement) 

Challenge 

Very  great,  great, 
or  moderate 
challenge 

Slight  or  none 

Not  applicable/no 
basis  to  judge 

Cultural  resistance  to  change 

71 

21 

8 

Parochialism  (i.e.,  focusing  on  one’s  own  sub-organization  rather 
than  having  an  enterprisewide  view.) 

67 

25 

8 

Availability  of  skilled  staff 

67 

25 

8 

Availability  of  training 

67 

25 

8 

Quality  of  business  process  reengineering  compliance  guidance 

63 

33 

4 

Business  process  reengineering  compliance  review  process 

63 

33 

4 

Timing  of  business  process  reengineering  relative  to  system 
development  work 

63 

21 

17 

Source:  GAO  analysis  of  DOD  survey  responses.  |  GAO-15-627. 

Note:  Numbers  might  not  add  to  100  percent  due  to  rounding 


Office  of  DCMO  Is  Taking 
Steps  to  Improve  Business 
Process  Reengineering 
Efforts,  but  Survey  Results 
Suggest  Additional  Action 
May  Be  Needed 


DOD  has  taken  steps  to  improve  its  reengineering  efforts  that  may,  in 
part,  address  some  of  the  challenges  identified  in  our  survey  results.  With 
regard  to  parochialism  (i.e.,  focusing  on  one’s  own  sub-organization 
rather  than  having  an  enterprisewide  view),  the  department  is  developing 
online  tools  that  provide  additional  information  to  program  managers, 
portfolio  managers,  pre-certification  authorities,  and  the  Defense 
Business  Council.  For  example,  the  department’s  problem  statement 
portal  is  to  be  a  repository  for  problem  statement  submissions  and  is  to 
be  available  departmentwide.  In  addition,  the  department  has  developed 
its  Integrated  Business  Framework-Data  Alignment  Portal,  which  is  to 
provide,  among  other  things,  additional  information  about  individual 
business  systems,  such  as  information  about  which  systems  execute 
specific  business  activities  and  system  functions. 


Further,  with  respect  to  addressing  the  challenge  associated  with  the 
business  process  reengineering  compliance  review  process,  the 
department  has  taken  steps  to  help  ensure  improved  accountability  for  a 
portion  of  certification  and  approval  requests.  In  particular,  according  to 
officials  from  the  Office  of  the  DCMO,  the  DCMO  allowed  the  military 
departments  more  autonomy  and  responsibility  for  reviewing  their  system 
portfolios  during  fiscal  year  2015  certification  and  approval  reviews. 
Nevertheless,  as  we  have  previously  reported,  and  as  discussed  in 
appendix  II,  this  process  is  not  guided  by  specific  criteria  for  elevating 
certain  systems  to  the  Defense  Business  Council  that  might  require 
additional  oversight. 
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Notwithstanding  these  improvement  efforts,  as  reported  in  feedback  by 
the  military  department  portfolio  managers,  additional  work  is  needed. 
These  managers  provided  a  number  of  suggestions  to  help  address  the 
identified  challenges.  Suggestions  included: 

•  Improve  business  process  reengineering  training:  Two  portfolio 
managers  offered  suggestions  that  relate  to  improved  training.  For 
example,  one  manager  stated  that  the  department  should  establish 
minimum  training  standards. 

•  Improve  business  process  reengineering  guidance:  Two 

managers  offered  suggestions  associated  with  improved  guidance. 

For  example,  one  portfolio  manager  stated  that  sufficient  guidance 
does  not  exist  to  describe  meaningful  business  process  models  or 
how  such  models  should  be  analyzed. 

•  Align  business  process  reengineering  with  system  development 
activities:  One  portfolio  manager  stated  that  the  reengineering 
process  should  be  more  closely  tied  to  acquisition  milestones  instead 
of  being  assessed  on  an  annual  basis. 

According  to  GAO’s  standards  for  internal  controls, management  should 
ensure  that  there  are  adequate  means  of  obtaining  information  from 
stakeholders  that  may  have  a  significant  impact  on  the  agency  achieving 
its  goals.  While  we  did  not  evaluate  the  effectiveness  of  these 
suggestions,  they  may  be  valuable  for  the  Office  of  the  DCMO  to  consider 
in  its  ongoing  and  future  business  process  reengineering  improvement 
efforts.  More  fully  addressing  the  challenges  cited  by  the  portfolio 
managers  would  help  the  department  achieve  better  outcomes,  including 
limiting  the  tailoring  of  commercial  off-the-shelf  systems. 


Conclusions 


DOD  has  made  progress  in  improving  its  compliance  with  section  332  of 
the  NDAA  for  fiscal  year  2005,  as  amended.  Specifically,  the  department 
has  implemented  5  of  the  16  recommendations  that  we  have  made  since 
201 1  that  are  consistent  with  the  requirements  of  the  act  and  has  partially 
implemented  the  remaining  1 1  recommendations.  The  recommendations 
not  fully  implemented  relate  to  improving  the  department’s  investment 
management  processes  and  efforts  to  certify  defense  business  systems. 


^^GAO,  Auditing  and  Financiai  Management:  Standards  for  Internat  Controt  in  the  Federat 
Government,  A\MD-00-2^  .3 A  (Washington,  D.C.:  Nov.  1,  1999). 
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among  other  things.  Fully  implementing  them  will  help  improve  DOD’s 
modernization  management  controls  and  fulfill  the  department’s  execution 
of  the  act’s  requirements. 

Collectively,  DOD’s  business  enterprise  architecture  and  business 
process  reengineering  efforts  show  mixed  results  in  their  effectiveness 
and  usefulness  in  achieving  the  intended  outcomes  and  benefits.  Among 
other  things,  portfolio  managers  reported  that  the  architecture  does  not 
enable  DOD  to  produce  reliable  and  timely  information  for  decision¬ 
making  purposes.  Additionally,  DOD’s  reengineering  efforts  are  effective 
in  streamlining  business  processes,  but  not  in  reducing  the  tailoring  of 
commercial  software  products.  Portfolio  managers  reported  that  various 
challenges  exist  in  achieving  intended  outcomes  and  benefits,  including 
cultural  resistance,  parochialism,  and  a  lack  of  skilled  staff.  DOD  has 
various  improvement  efforts  under  way;  however,  gaps  exist  and  portfolio 
managers  provided  suggestions  on  how  to  close  some  of  them.  Until 
these  gaps  are  addressed,  the  department’s  ability  to  achieve  important 
outcomes  and  benefits  will  continue  to  be  limited. 


Recommendation  for 
Executive  Action 


To  help  ensure  that  the  department  can  better  achieve  business  process 
reengineering  and  enterprise  architecture  outcomes  and  benefits,  we 
recommend  that  the  Secretary  of  Defense  utilize  the  results  of  our 
portfolio  manager  survey  to  determine  additional  actions  that  can  improve 
the  department’s  management  of  its  business  process  reengineering  and 
enterprise  architecture  activities. 


Agency  Comments 
and  Our  Evaluation 


We  received  written  comments  on  a  draft  of  this  report  from  DOD’s 
Deputy  Chief  Management  Officer  (DCMO).  The  comments  are  reprinted 
in  appendix  III. 

In  the  comments,  the  DCMO  concurred  with  our  recommendation  and 
stated  that  the  department  will  use  the  results  of  our  portfolio  manager 
survey  to  help  make  improvements.  The  DCMO  also  described 
associated  improvement  efforts.  For  example,  the  DCMO  stated  that  the 
department  plans  to  restructure  the  Business  Enterprise  Architecture  to 
focus  more  explicitly  on  the  business  processes  being  executed  within  the 
functional  domains,  which  span  all  levels  of  the  department.  DOD  officials 
also  provided  technical  comments,  which  we  have  incorporated  as 
appropriate. 
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We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees;  the  Director,  Office  of  Management  and  Budget;  the 
Secretary  of  Defense;  and  other  interested  parties.  This  report  also  is 
available  at  no  charge  on  the  GAO  website  at  http://www.gao.gov.  If  you 
or  your  staff  members  have  any  questions  on  matters  discussed  in  this 
report,  please  contact  me  at  (202)  512-4456  or  chac@gao.gov.  Contact 
points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs  may 
be  found  on  the  last  page  of  this  report.  GAO  staff  who  made  major 
contributions  to  this  report  are  listed  in  appendix  IV. 


Carol  R.  Cha 
Director 

Information  Technology  Acquisition  Management  Issues 
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List  of  Committees 


The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  J.  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


Our  objectives  were  to  (1)  assess  the  actions  by  the  Department  of 
Defense  to  comply  with  section  332  of  the  National  Defense  Authorization 
Act  (NDAA)  for  Fiscal  Year  2005,  as  amended^  and  (2)  determine  the 
usefulness  and  effectiveness  of  DOD’s  business  enterprise  architecture 
and  business  process  reengineering  processes. ^ 

To  address  the  first  objective,  we  identified  recommendations  related  to 
DOD’s  business  systems  modernization  efforts  that  we  made  in  our 
annual  reports  from  201 1  to  2014  (16  recommendations  total)  in  response 
to  the  fiscal  year  2005  NDAA’s  requirements.  Though  we  have  made 
recommendations  in  this  area  prior  to  201 1 ,  those  recommendations  have 
since  been  closed.  We  evaluated  the  department’s  written  responses  and 
related  documentation  on  steps  completed  to  implement  or  partially 
implement  the  recommendations.  Documentation  we  analyzed  included 
guidance  on  business  enterprise  architecture  and  business  process 
reengineering  compliance,  guidance  on  certifying  and  approving  defense 
business  systems,  and  documentation  about  the  department’s  problem 
statement  development  and  review  process.  In  addition,  we  interviewed 
officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  and  the 
Office  of  the  Chief  Information  Officer,  and  observed  a  demonstration  of 
the  Office  of  the  Deputy  Chief  Management  Officer’s  Integrated  Business 
Framework-Data  Alignment  Portal  tool  to  better  understand  the  actions 
taken  to  address  our  recommendations.  We  also  reviewed  the 
department’s  annual  report  to  Congress,  which  was  submitted  on  April  6, 
201 5,  to  identify  gaps  or  inconsistencies  with  the  implementation  of  the  1 6 
recommendations. 

To  address  our  second  objective,  we  determined  the  intended  outcomes 
of  the  business  enterprise  architecture  and  business  process 
reengineering  processes  by  analyzing  the  fiscal  year  2005  NDAA.  We 
also  determined  potential  benefits  associated  with  the  processes  by 


^  Pub.  L.  No.  1 08-375,  §  332,  1 1 8  Stat.  1 811 ,  1 851  -1 856  (2004),  as  amended  by  the 
NDAA  for  Fiscal  Year  2015,  Pub.  L  No.  113-291,  §  901,  128  Stat.  3292  (2014),  NDAA  for 
Fiscal  Year  2014,  Pub.  L  No.  113-66,  §  901,  127  Stat.  672,  815  (2013),  the  NDAA  for 
Fiscal  Year  201 3,  Pub.  L.  No.  112-239,  §  906,  126  Stat.  1632,  1869  (2012),  the  NDAA  for 
Fiscal  Year  2012,  Pub.  L  No.  112-81,  §  901,  125  Stat.  1298,  1527  (2011),  and  the  NDAA 
for  Fiscal  Year  2010,  Pub.  L.  No.  111-84,  §  1072,  123  Stat.  2190,  2470  (2009).  This 
provision  is  codified  in  part  at  10  U.S.C.  §  2222. 

^S.  Rep.  No.  113-176,  at  147  (2014),  accompanying  S.  2410  and  associated  with  FI.R. 
3979  that  became  Pub.  L.  No.  1 1 3-291 . 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


reviewing  department  guidance  on  the  processes  and  related 
documentation.  This  includes  DOD’s  business  enterprise  architecture  and 
business  process  reengineering  guidance,  Defense  Business  System 
Investment  Management  Process  Guidance,  the  Business  Case  Analysis 
Template,  DOD’s  Business  Enterprise  Architecture  10.0  AV-1  Overview 
and  Summary  Information,  the  department’s  Strategic  Management  Plan, 
and  the  Information  Resource  Management  Strategic  Plan  for  fiscal  years 
2014  and  2015.  We  also  reviewed  relevant  GAO  reports  on  business 
enterprise  architecture  and  business  process  reengineering.^ 

We  then  developed  a  structured  data  collection  instrument  (survey)  to 
gather  information  on  the  usefulness  of  the  two  specified  IT 
modernization  management  controls  at  DOD  in  achieving  their  intended 
outcomes  and  their  effectiveness  in  achieving  associated  benefits.  As 
part  of  this  survey,  we  also  developed  questions  to  help  us  determine  (1) 
challenges  related  to  complying  with  the  processes  and  (2)  suggestions 
for  achieving  business  enterprise  architecture  and  business  process 
reengineering  outcomes,  including  suggestions  for  achieving  these 
outcomes  in  a  more  cost-effective  manner. 

Selected  questions  contained  a  ratings  scale  for  managers  to  choose  a 
response  that  was  consistent  with  the  aforementioned  topic  areas.  For 
example,  we  asked  managers  to  rate  the  effectiveness  of  the  business 
enterprise  architecture  and  business  process  reengineering  efforts  using 
a  scale  containing  the  following  choices: 

•  very  effective, 

•  somewhat  effective, 

•  neither  effective  nor  ineffective, 

•  somewhat  ineffective, 

•  very  ineffective,  or 

•  not  applicable/no  basis  to  judge. 


^See,  for  example,  GAO,  Organizational  Transformation:  Enterprise  Architecture  Value 
Needs  to  Be  Measured  and  Reported.  GAO-12-791  (Washington,  D.C.:  Sept.  26,  2012); 
Organizational  Transformation:  A  Framework  for  Assessing  and  Improving  Enterprise 
Architecture  Management  (Version  2.0).  GAO-1 0-846G  (Washington,  D.C.:  Aug  6,  2010); 
and  Business  Process  Reengineering  Assessment  Guide  (Version  3),  GAO/AIMD-10.1.15 
(Washington,  D.C.:  May  1997). 
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We  also  asked  managers  to  identify  the  extent  to  which  their  portfolios 
had  achieved  benefits  associated  with  business  enterprise  architecture 
and  business  process  reengineering  efforts  using  a  scale  containing  the 
following  choices: 

•  very  great  extent, 

•  great  extent, 

•  moderate  extent, 

•  some  extent, 

•  little  or  no  extent,  or 

•  not  applicable/no  basis  to  judge. 

We  pre-tested  the  questions  with  various  DOD  officials  including  officials 
from  the  Office  of  the  Deputy  Chief  Management  Officer,  and  with 
portfolio  and  program-level  officials  within  the  military  departments.  As  a 
result,  we  determined  that  the  military  department  portfolio  managers 
were  in  the  best  position  to  answer  our  questions  because  they  manage 
and  have  a  perspective  across  an  entire  portfolio  of  defense  business 
systems.  Officials  from  DCMO’s  Management,  Policy,  and  Analysis 
Directorate  provided  us  with  a  list  of  portfolio  managers  for  the  three 
military  departments.  We  did  not  include  portfolio  managers  for  DOD 
entities  outside  of  the  military  departments. 

We  obtained  responses  from  all  surveyed  portfolio  managers  (24  in 
total). Accordingly,  these  results  are  generalizable.  We  analyzed  and 
summarized  the  survey  results  to  help  determine  the  usefulness  and 
effectiveness  of  DOD’s  business  process  reengineering  and  enterprise 
architecture  efforts,  as  well  as  related  challenges  and  suggestions  for 
improvement.  In  addition,  though  we  collected  examples  of  cost  savings 
estimates  from  managers,  and  cite  them  in  the  report,  we  did  not  assess 
the  cited  cost  savings  estimates. 


^Initially,  27  portfolio  managers  were  identified  as  respondents.  However,  3  managers 
were  excluded  from  the  respondent  list  at  DOD’s  request  for  various  reasons.  For 
example,  there  were  multiple  managers  that  had  been  identified  for  one  portfolio,  and  to 
avoid  duplicate  responses,  we  asked  that  only  1  manager  respond  for  that  particular 
portfolio. 
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We  also  met  with  managers  of  selected  DOD  business  system  programs 
and  other  knowledgeable  DOD  officials  to  discuss  their  perspectives  on 
DOD’s  business  enterprise  architecture  and  business  process 
reengineering  efforts.  This  included  interviewing  officials  associated  with 
defense  business  programs  from  each  of  the  military  departments  and 
from  across  various  business  functions,  including  program  managers, 
enterprise  architects,  and  other  technical  and  program  operations 
officials.  Further,  when  available,  we  reviewed  documentation  provided  by 
DOD  program  managers  to  substantiate  answers  provided  as  part  of  our 
interviews.  We  also  discussed  the  survey  results  with  officials  from  the 
Office  of  the  DCMO  to  obtain  their  perspectives  on  the  results  and 
discussed  with  these  officials  ongoing  efforts  to  improve  the  department’s 
business  process  reengineering  and  enterprise  architecture  efforts. 

The  practical  difficulties  of  conducting  any  survey  may  introduce  errors, 
commonly  referred  to  as  nonsampling  errors.  For  example,  difficulties  in 
how  a  particular  question  is  interpreted,  in  the  sources  of  information  that 
are  available  to  respondents,  or  in  how  the  survey  data  are  analyzed  can 
all  introduce  unwanted  variability  into  survey  results.  To  minimize  such 
nonsampling  errors,  a  social  science  survey  specialist  designed  the 
questionnaire  in  collaboration  with  GAO  staff  with  subject  matter 
expertise.  As  stated  earlier,  the  questionnaire  was  pre-tested  to  ensure 
that  the  questions  were  relevant,  clearly  stated,  and  easy  to  comprehend. 
When  data  from  the  survey  were  analyzed,  an  independent  analyst 
reviewed  the  computer  program  used  for  the  analysis  of  the  survey  data. 
Since  this  was  a  web-based  survey,  respondents  entered  their  answers 
directly  into  the  electronic  questionnaire,  thereby  eliminating  the  need  to 
have  the  data  keyed  into  a  database  and  avoiding  data  entry  errors. 

We  conducted  this  performance  audit  from  October  2014  to  July  2015  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objective. 
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Table  10  describes  the  status  of  open  GAO  recommendations  associated 
with  the  fiscal  year  2005  NDAA’s  requirement  that  we  annually  assess  the 
actions  taken  by  the  department  to  comply  with  its  provisions.  Since  June 
2011,  we  have  made  16  recommendations  to  DOD  regarding  defense 
business  systems.  As  of  April  2015,  the  department  had  implemented  5, 
and  partially  implemented  11  recommendations.  The  table  also  identifies 
the  category  that  we  assigned  to  the  recommendation  to  demonstrate  its 
relationship  to  the  requirements  outlined  in  the  act. 


Table  10:  Status  of  Recommendations  Made  Since  2011 


GAO  report,  recommendation, 
and  category 

Implemented 

Partially 

implemented 

GAO  assessment 

GAO-1 1-684:  Department  of  Defense: 
Further  Actions  Needed  to  Institutionalize 
Key  Business  System  Modernization 
Management  Controls,  June  29,  2011 

1 .  The  Secretary  of  Defense  should 
expeditiously  complete  the 
implementation  of  the  announced 
transfer  of  functions  of  the  Business 
Transformation  Agency  and  the  Office  of 
the  Assistant  Secretary  of  Defense  for 
Networks  and  Information 
Integration/Department  of  Defense  Chief 
Information  Officer  and  provide  specificity 
as  to  when  and  where  these  functions 
will  be  transferred.  (Business  Enterprise 
Architecture) 

X 

As  we  reported  in  May  2013,®  the  department  formally 
disestablished  the  Business  Transformation  Agency  in 
October  201 1 ,  completing  the  transfer  of  its  various 
functions  to  other  DOD  entities,  including  the  Office  of 
the  Deputy  Chief  Management  Officer  (DCMO).  In 
addition,  in  January  2012,  DOD  announced  the 
disestablishment  of  the  Assistant  Secretary  of  Defense 
for  Networks  and  Information  Integration  and  the 
transfer  of  its  various  functions  to  other  DOD  entities, 
including  the  DOD  Chief  Information  Officer  and  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics. 
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GAO  report,  recommendation,  Partially 

and  category  Implemented  implemented  GAO  assessment 

GAO-12-685:  DOD  Business  Systems 
Modernization:  Governance  Mechanisms 
for  impiementing  Management  Controis 
Need  to  Be  improved,  June  1,  2012 


1 .  The  Secretary  of  Defense  should  ensure 
that  the  Deputy  Secretary  of  Defense,  as 
the  department’s  Chief  Management 
Officer,  establish  a  policy  that  clarifies 
the  roles,  responsibilities,  and 
relationships  among  the  Chief 
Management  Officer,  Deputy  Chief 
Management  Officer  (DCMO),  DOD  and 
military  department  Chief  Information 
Officers,  Principal  Staff  Assistants, 
military  department  Chief  Management 
Officers,  and  the  heads  of  the  military 
departments  and  defense  agencies, 
associated  with  the  development  of  a 
federated  business  enterprise 
architecture. 

Among  other  things,  the  policy  should 
address  the  development  and 
implementation  of  an  overarching 
taxonomy  and  associated  ontologies  to 
help  ensure  that  each  of  the  respective 
portions  of  the  architecture  will  be 
properly  linked  and  aligned. 

In  addition,  the  policy  should  address 
alignment  and  coordination  of  business 
process  areas,  military  department  and 
defense  agency  activities  associated  with 
developing  and  implementing  each  of  the 
various  components  of  the  Business 
Enterprise  Architecture,  and  relationships 
among  these  entities.  (Business 
Enterprise  Architecture) 


X  The  department  has  taken  steps  to  address  the  intent 

of  this  recommendation.  For  example,  the  DCMO 
approved  the  Business  Enterprise  Architecture 
Configuration  Control  Board  charter  on  August  19, 
2013.  The  board  includes  the  Business  Enterprise 
Architecture  Chief  Architect  and  representatives  from 
both  the  department’s  Office  of  the  Chief  Information 
Officer  as  well  as  the  Chief  Management  Officer 
organizations  of  the  Defense  Business  Council 
member  organizations.  According  to  its  charter,  the 
board  is  to  be  the  principal  body  for  managing  the 
disposition  of  proposed  architecture  change  requests. 
The  board  is  to  be  supported  by  the  Component 
Collaboration  Forum,  which  is  to  focus  on  developing  a 
path  ahead  for  a  federated  business  architecture. 

The  department  has  also  developed  a  draft  plan  for  a 
federated  architecture.  Flowever,  the  plan  does  not 
provide  details  on  how  DOD  intends  to  extend 
architecture  content  across  the  department,  including 
the  military  departments.  In  addition,  the  department 
has  not  provided  details  of  an  overarching  taxonomy  to 
be  used  across  the  enterprise  or  established  a  policy 
that  clarifies  roles,  responsibilities,  and  relationships 
as  called  for  by  our  recommendation. 
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and  category  Implemented  implemented  GAO  assessment 


2.  The  Secretary  of  Defense  should  direct  X 

the  appropriate  DOD  organizations  to 
establish  a  deadline  by  which  it  intends 
to  complete  the  integration  of  the 
repositories  and  validate  the 
completeness  and  reliability  of 
information.  (Mandated  Budget 
Reporting) 


The  department  has  taken  steps  to  implement  this 
recommendation.  In  particular,  in  2013,  the  Office  of 
the  DCMO  established  the  DOD  Information 
Technology  Investment  Portal  to  serve  as  the 
authoritative  data  source  for  Defense  Business 
Systems  certification  funding  and  approval  information. 
In  addition,  the  department  has  established  common 
elements  in  its  three  primary  repositories  used  for 
tracking  information  about  business  systems — DOD 
Information  Technology  Investment  Portal,  Department 
of  Defense  Information  Technology  Portfolio 
Repository,  and  Select  &  Native  Programming  Data 
Input  System  for  Information  Technology — that  allow 
information  about  individual  business  systems  to  be 
integrated  across  the  repositories. 

Moreover,  the  Office  of  the  Chief  Information  Officer 
demonstrated  that  it  conducts  periodic  data  quality 
assessments.  For  example,  the  results  of  the  most 
recent  assessment  provided  by  DOD  demonstrate  that 
the  number  of  business  systems  is  generally 
consistent  across  its  repositories. 


3.  The  Secretary  of  Defense  should  ensure 
that  the  Deputy  Secretary  of  Defense,  as 
the  department’s  Chief  Management 
Officer,  direct  the  Deputy  Chief 
Management  Officer  to  include  in  DOD’s 
annual  report  to  Congress  on  compliance 
with  10  U.S.C.  §  2222  the  results  of  the 
department’s  business  process 
reengineering  efforts. 

Among  other  things,  the  results  should 
include  the  department’s  determination  of 
the  number  of  systems  that  have 
undergone  material  process  changes, 
the  number  of  interfaces  eliminated  as 
part  of  these  efforts  (i.e.,  by  program,  by 
name),  and  the  status  of  its  end-to-end 
business  process  reengineering  efforts. 
(Investment  Certification  and  Approval) 


X  The  Office  of  the  DCMO’s  2015  Congressional  Report 

on  Defense  Business  Operations  included  some 
information  about  its  business  process  reengineering 
efforts,  but  the  report  did  not  include  the  department’s 
determination  of  the  number  of  systems  that  have 
undergone  material  process  changes,  the  number  of 
interfaces  eliminated  as  part  of  these  efforts  (i.e.,  by 
program,  by  name),  and  the  status  of  its  end-to-end 
business  process  reengineering  efforts.  For  example, 
the  department’s  report  to  Congress  stated  that  the 
Army  utilized  business  process  reengineering  as  part 
of  a  personnel  and  pay  program  to  reengineer  157 
discrete  personnel  processes  to  fit  the  capabilities  of  a 
commercial  enterprise  resource  planning  system. 
While  the  department’s  annual  report  included 
information  about  specific  efforts,  the  Office  of  the 
DCMO  has  not  yet  reported  on  measures  such  as 
those  called  for  by  our  recommendation. 

According  to  officials  from  the  Office  of  the  DCMO,  its 
annual  report  is  not  intended  to  provide  the  level  of 
detail  requested  by  this  recommendation.  Further, 
these  officials  stated  that  the  Office  of  the  DCMO  does 
not  perform  business  process  reengineering 
assessments.  Rather,  the  pre-certification  authorities 
have  the  responsibility  to  perform  business  process 
reengineering.  Nevertheless,  regardless  of  who 
conducts  business  process  reengineering,  the 
department  has  not  demonstrated  that  it  has  reported 
on  the  results  of  business  process  reengineering 
efforts  as  called  for  by  our  recommendation  in  either 
its  annual  report  or  in  any  other  report  to  Congress. 
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and  category  Implemented  implemented  GAO  assessment 


4.  The  Deputy  Secretary  of  Defense,  as  the 
department’s  Chief  Management  Officer, 
should  direct  the  Deputy  Chief 
Management  Officer  to  include  in  DOD’s 
annual  report  to  Congress  on  compliance 
with  10  U.S.C.  §  2222,  an  update  on  the 
office  of  the  DCMO’s  progress  toward 
filling  staff  positions  and  the  impact  of 
any  unfilled  positions  on  the  ability  of  the 
office  to  conduct  its  work.  (Other  - 
Human  Capital) 


X  The  Office  of  the  DCMO  provided  an  update  to  GAO 

on  the  numbers  of  positions  filled  and  open.  In 
addition,  officials  provided  documentation  associated 
with  the  department’s  fiscal  year  2016  budget  request 
identifying  information  about  changes  in  requested 
funds  and  full-time  equivalent  positions.  Officials  also 
identified  examples  of  publicly  available  information 
about  full-time  equivalent  positions  that  the  office  was 
seeking  to  fill.  Nevertheless,  an  update  on  staffing  and 
the  impact  of  unfilled  positions  on  the  ability  of  the 
office  to  conduct  its  work  has  not  yet  been  included  in 
the  annual  report  or  in  other  reports  to  Congress. 


1 .  The  Secretary  of  Defense  should  direct 
the  Deputy  Chief  Management  Officer  to 
define  by  when  and  how  the  department 
plans  to  develop  an  architecture  that 
would  extend  to  all  defense  components 
and  include,  among  other  things: 

(a)  information  about  the  specific 
business  systems  that  support  business 
enterprise  architecture  business  activities 
and  related  system  functions, 

(b)  business  capabilities  for  the  Hire-to- 
Retire  and  Procure-to-Pay  business 
processes,  and 

(c)  sufficient  information  about  business 
activities  to  allow  for  more  effective 
identification  of  potential  overlap  and 
duplication.  (Business  Enterprise 
Architecture) 


The  department  has  taken  steps  to  improve  the 
integration  of  Business  Enterprise  Architecture 
information  with  other  existing  information.  For 
example,  this  integration  is  to  allow  the  department  to 
identify  information  such  as  mapping  of  existing 
business  systems  to  individual  Business  Enterprise 
Architecture  system  functions.  In  addition,  officials 
from  the  office  of  the  DCMO  provided  a  draft  plan  for 
business  enterprise  architecture  federation,  which 
includes  steps  associated  with  extending  the 
architecture  to  all  defense  components.  Nevertheless, 
officials  stated  that  the  plan  is  not  yet  complete. 

Moreover,  the  department  has  yet  to  define  by  when 
and  how  it  will  develop  an  architecture  that  extends  to 
all  defense  components.  Officials  from  the  Office  of  the 
DCMO  stated  that  they  are  working  to  federate  Army 
business  architecture  information  into  the  Business 
Enterprise  Architecture  and  that  this  effort  will  inform 
future  steps;  however,  that  effort  remains  a  work  in 
progress. 


GAO-13-557:  DOD  Business  Systems 
Modernization:  Further  Actions  Needed  to 
Address  Chaiienges  and  improve 
Accountabiiity,  May  17,  2013 

X 
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and  category  Implemented  implemented  GAO  assessment 


2.  The  Secretary  of  Defense  should  direct 
the  Deputy  Chief  Management  Officer  to 
define  by  when  and  how  the  enterprise 
transition  plan  will  include,  among  other 
things, 

(a)  milestones,  performance  measures, 
and  funding  plans  for  all  business 
systems  expected  to  be  part  of  the  target 
architecture  and  each  system’s  risks  or 
challenges  to  integration;(b)  time-phased 
end  dates  associated  with  terminating 
legacy  systems  in  phases;(c)  a  listing  of 
all  other  defense  business  systems 
(including  systems  that  are  considered  to 
be  core  systems)  that  will  be  a  part  of  the 
target  defense  business  systems 
computing  environment  and  a  strategy 
for  making  modifications  to  those 
systems  that  will  be  needed  to  ensure 
that  they  comply  with  the  defense 
business  enterprise  architecture, 
including  time-phased  milestones, 
performance  measures,  and  financial 
resource  needs;  and(d)  information  about 
how  systems  are  to  be  sequenced 
according  to,  among  other  things, 
dependencies  among  investments. 
(Enterprise  Transition  Plan) 


X  The  Office  of  the  DCMO  has  taken  steps  to  improve 

information  available  about  its  business  systems.  More 
recently,  the  department  has  established  the 
Integrated  Business  Framework-Data  Alignment  Portal 
as  a  repository  of  automated  information  available 
about  Functional  Strategies,  Organizational  Execution 
Plans,  and  the  business  architecture.  According  to 
officials  from  the  Office  of  the  DCMO,  this  portal  will  be 
used  to  document,  among  other  things,  the  data  that 
are  to  be  included  in  Functional  Strategies  and 
Organizational  Execution  Plans.  Such  a  collective  set 
of  data  may  be  used  to  generate  transition  plan 
information.  Flowever,  the  full  implementation  of  this 
new  approach  remains  to  be  seen.  Moreover,  the 
department  has  not  provided  a  plan  that  defines  when 
and  how  it  will  address  the  various  elements  called  for 
in  our  recommendation. 


3.  The  Secretary  of  Defense  should  direct 
the  Deputy  Chief  Management  Officer  to 
ensure  that  the  functional  strategies 
include  all  of  the  critical  elements 
identified  in  DOD  investment 
management  guidance,  including 
performance  measures  to  determine 
progress  toward  achieving  the  goals  that 
incorporate  all  of  the  attributes  called  for 
in  the  department’s  guidance. 
(Investment  Management) 


X  DOD  established  performance  measures  in  its 

functional  strategies  that  addressed  at  least  some  of 
the  five  attributes  called  for  in  DOD  guidance.  For 
example,  all  of  the  fiscal  year  201 5  functional 
strategies  identified  examples  of  quantitative  metrics. 
Flowever,  not  all  functional  strategies  identified  metrics 
that  addressed  the  other  attributes.  Specifically,  the 
strategies  did  not  all  include  performance  measures 
that  addressed  the  following  attributes:  (1 )  data  that 
are  tracked  incrementally  over  a  specified  period,  (2)  a 
baseline  for  each  performance  measure,  (3)  a  target 
against  the  baseline,  and  (4)  a  rationale  for  the 
identified  target.  In  addition,  DOD’s  fiscal  year  2016 
functional  strategies  also  lacked  such  attributes. 
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4.  The  Secretary  of  Defense  should  direct 
the  Deputy  Chief  Management  Officer  to 
select  and  control  its  mix  of  investments 
in  a  manner  that  best  supports  mission 
needs  by 

(a)  documenting  a  process  for  evaluating 
portfolio  performance  that  includes  the 
use  of  actual  versus  expected 
performance  data  and  predetermined 
thresholds; 

(b)  ensuring  that  portfolio  assessments 
are  conducted  in  key  areas  identified  in 
our  IT  investment  management 
framework:  benefits  attained;  current 
schedule;  accuracy  of  project  reporting; 
and  risks  that  have  been  mitigated, 
eliminated,  or  accepted  to  date;  and 

(c)  ensuring  that  the  documents  provided 
to  the  Defense  Business  Council  as  part 
of  the  investment  management  process 
include  critical  information  for  conducting 
all  assessments.  (Investment 
Management) 


X  The  department’s  February  2015  investment 

management  guidance  identifies  four  criteria  and 
specifies  the  associated  assessments  that  are  to  be 
conducted  when  reviewing  and  evaluating  component- 
level  organizational  execution  plans  in  order  to  make  a 
portfolio-based  investment  decision.  The  guidance 
also  provides  additional  details  regarding  considering 
return  on  investment  when  assessing  program  costs. 

In  addition,  the  guidance  states  that  organizational 
execution  plans  will  be  assessed  from  various 
perspectives,  including  progress  toward  the  target 
environment,  business  value,  cost,  and  risk. 
Nevertheless,  the  guidance  does  not  specify  a  process 
for  conducting  an  assessment  or  call  for  the  use  of 
actual  versus  expected  performance  data  and 
predetermined  thresholds.  In  addition,  the  guidance 
does  not  call  for  documents  provided  to  the  Defense 
Business  Council  to  include  critical  information  for 
conducting  assessments,  such  as  information  about 
system  scalability  to  support  additional  users  or  new 
features  in  the  future  and  cost  in  relationship  to  return 
on  investment. 
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GAO  report,  recommendation,  Partially 

and  category  Implemented  implemented  GAO  assessment 


5.  The  Secretary  of  Defense  should  direct 
the  Deputy  Chief  Management  Officer  to 
implement  and  use  the  business 
enterprise  architecture  and  business 
process  reengineering  compliance 
assessments  more  effectively  to  support 
organizational  transformation  efforts  by 

(a)  disclosing  relevant  information  about 
known  weaknesses,  such  as  business 
enterprise  architecture  and  business 
process  reengineering  compliance 
weaknesses  for  systems  that  were  not 
certified  or  certified  with  qualifications  in 
annual  reports  to  Congress; 

(b)  establishing  milestones  by  which 
selected  validations  of  business 
enterprise  architecture  compliance 
assertions  are  to  be  completed;  and 

(c)  ensuring  that  appropriate  business 
process  reengineering  assertions  have 
been  completed  on  all  investments 
submitted  for  the  fiscal  year  2014 
certification  reviews  prior  to  the 
certification  of  funds.  (Investment 
Certification  and  Approval) 


X  The  2015  Congressional  Report  on  Defense  Business 

Operations  included  some  information  consistent  with 
our  recommendation.  For  example,  it  contained 
information  about  weaknesses  for  systems  that  were 
certified  with  qualifications.  In  particular,  the  report 
stated  that  the  department  conditionally  approved  29 
military  department  and  30  defense  agency  requests 
pending  Defense  Business  Council  approval  of  their 
problem  statements.  The  report  also  cited  the  specific 
systems  that  were  conditionally  approved  pending 
approval  of  their  problem  statements.  Nevertheless,  it 
did  not  disclose  the  results  of  business  enterprise 
architecture  validations  that  were  to  occur  as  part  of 
the  certification  and  approval  process  for  fiscal  year 
2015. 

In  addition,  according  to  the  April  2014  guidance  for 
investment  certification  and  approval,  upon  receipt  of 
an  organization’s  Organizational  Execution  Plan,  the 
defense  business  council  chair  was  to  identify 
generally  no  more  than  three  defense  business 
systems  to  be  assessed  from  a  component’s  portfolio. 
Once  notified,  the  pre-certification  authority  was  to 
have  5  working  days  to  provide  the  assessment 
documentation  used  to  assert  business  process 
reengineering  or  business  enterprise  architecture 
compliance.  However,  the  guidance  does  not  specify 
time  frames  or  milestones  for  completing  these 
validations. 

The  Office  of  the  DCMO  also  provided  data  from  the 
system  that  maintains  certification  and  approval 
information.  These  data  showed  that  only  two  systems 
were  certified  and  approved  for  fiscal  year  2014 
without  a  business  process  reengineering  assertion. 
The  office  also  provided  additional  information  about 
these  systems  explaining  the  rationale  for  not 
conducting  business  process  reengineering. 


6.  The  Secretary  of  Defense  should  direct 
the  Deputy  Chief  Management  Officer  to 
develop  a  skills  inventory,  needs 
assessment,  gap  analysis,  and  plan  to 
address  identified  gaps  as  part  of  a 
strategic  approach  to  human  capital 
planning  for  the  Office  of  the  Deputy 
Chief  Management  Officer.  (Other  - 
Human  Capital) 


X  The  Office  of  the  DCMO  has  taken  some  steps  to 

address  this  recommendation.  For  example,  it  has 
developed  a  draft  resource  allocation  plan,  which 
identifies  staffing  profiles  for  each  of  the  office’s 
directorates  and  their  respective  divisions.  These 
profiles  cite  needed  staff  competencies  and 
qualifications.  However,  the  department  has  not 
developed  a  skills  inventory,  gap  analysis,  or  plan  to 
address  identified  gaps  as  part  of  a  strategic  approach 
to  human  capital  planning. 
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GAO  report,  recommendation,  Partially 

and  category  Implemented  implemented  GAO  assessment 


7.  The  Secretary  of  Defense  should  direct 
the  appropriate  authority  to  ensure  that 
complete  documentation,  such  as  root 
cause  analyses,  assessments  of  existing 
interfaces  for  reuse  opportunities,  and 
performance  metrics  related  to  the 
reengineering  efforts,  is  provided  as  part 
of  the  fiscal  year  2014  certification  and 
approval  process  for  the  Integrated 
Personnel  and  Pay  System  -  Army 
(IPPS-A),  Integrated  Personnel  and  Pay 
System  -  Navy  (IPPS-N),  Air  Force 
Integrated  Personnel  and  Pay  System 
(AF-IPPS),  and  Integrated  Electronic 
Health  Record  (iEHR)  investments. 
(Investment  Certification  and  Approval) 


X  DOD  has  taken  some  steps  to  address  this 

recommendation.  For  example,  the  department 
demonstrated  that  it  had  completed  documentation, 
such  as  root  cause  analyses,  assessments  of  existing 
interfaces  for  reuse  opportunities,  and  performance 
metrics  related  to  the  reengineering  efforts,  and  that 
the  documentation  was  provided  as  part  of  the 
certification  and  approval  process  for  the  Air  Force 
Integrated  Personnel  and  Pay  System  investment. 
However,  it  did  not  demonstrate  that  such 
documentation  was  fully  completed  and  provided  as 
part  of  the  certification  and  approval  process  for  other 
systems.  For  example,  DOD  only  demonstrated  that 
partial  documentation  had  been  completed  and 
provided  for  the  Integrated  Electronic  Health  Record 
investment. 


8.  The  Secretary  of  Defense  should  direct  X 

the  appropriate  authority  to  determine 
whether  funds  were  properly  obligated 
under  1 0  U.S.C.  2222(a)-(b)  for  systems 
for  which  appropriate  business  process 
reengineering  assertions  were  not 
completed.  (Investment  Certification  and 
Approval) 


Officials  from  the  Office  of  the  DCMO  demonstrated 
that  the  department  has  addressed  the  intent  of  this 
recommendation.  Specifically,  while  the  department 
did  not  concur  with  the  recommendation  and  did  not 
make  the  recommended  determination,  it  has  taken 
mitigating  steps  to  help  ensure  compliance  with 
business  process  reengineering  requirements.  For 
example,  officials  stated  that,  as  part  of  the  fiscal  year 
2013  certification  and  approval  process,  conditions 
were  imposed  by  the  investment  review  board 
requiring  all  components  to  submit  a  plan  on  how  core 
defense  business  systems  would  become  compliant 
with  the  act’s  business  process  reengineering 
requirement.  These  officials  also  provided 
documentation  showing  that  the  department  tracked 
these  conditions. 

In  addition,  the  department  has  reported  much  higher 
levels  of  compliance  with  the  act’s  business  process 
reengineering  requirements  in  subsequent  annual 
review  cycles.  For  example,  in  May  2013,  we  reported 
that,  according  to  DOD,  appropriate  business  process 
reengineering  had  been  undertaken  on  only  about  41 
percent  of  the  approximately  1 ,200  systems  for  the 
fiscal  year  2013  certification  reviews.  In  contrast, 
officials  from  the  Office  of  the  DCMO  stated  that  only  2 
systems  were  certified  and  approved  during  the  fiscal 
year  2014  certification  and  approval  cycle  and  6 
systems  were  certified  and  approved  during  the  fiscal 
year  2015  certification  and  approval  cycle  that  did  not 
have  complete  business  process  reengineering 
assertions.  Moreover,  these  officials  provided 
justifications  for  why  each  of  these  systems  did  not 
have  complete  business  process  reengineering 
assertions. 
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GAO  report,  recommendation,  Partially 

and  category  Implemented  implemented  GAO  assessment 


GAO-14-486:  Defense  Business  Systems: 
Further  Refinements  Needed  to  Guide 
the  Investment  Management  Process, 

May  12,  2014 

1 .  The  Secretary  of  Defense  should  direct  X 
the  appropriate  DOD  management  entity 
to  define  by  when  and  how  the 
department  plans  to  align  its  business 
system  certification  and  approval  process 
with  its  Planning,  Programming, 

Budgeting,  and  Execution  process. 
(Investment  Certification  and  Approval) 


DOD  has  taken  steps  to  align  its  business  system 
certification  and  approval  process  with  its  Planning, 
Programming,  Budgeting,  and  Execution  process.  For 
example,  according  to  the  department’s  February  2015 
certification  and  approval  guidance.  Organizational 
Execution  Plans  are  to  include  information  about 
certification  requests  for  the  upcoming  fiscal  year  as 
well  as  over  the  course  of  the  Future  Years  Defense 
Program.  All  of  this  information  is  to  be  considered 
when  making  certification  and  approval  decisions.  In 
addition,  the  guidance  states  that  the  chair  of  the 
Defense  Business  Council  will  make  programming  and 
budgeting  recommendations  to  the  Office  of  Cost 
Assessment  and  Program  Evaluation  and  the  DOD 
Comptroller. 


2.  The  Secretary  of  Defense  should  direct  X 

the  appropriate  DOD  management  entity 
to  define  criteria  for  reviewing  defense 
business  systems  at  an  appropriate  level 
in  the  department  based  on  factors  such 
as  complexity,  scope,  cost,  and  risk,  in 
support  of  the  certification  and  approval 
process.  (Investment  Management) 


According  to  officials  from  the  Office  of  the  DCMO,  the 
Defense  Business  Council  primarily  focused  its 
attention  on  the  non-military  department  business 
systems  during  the  fiscal  year  2015  certification  and 
approval  process.  Accordingly,  the  council  relied  on 
military  department  precertification  authority  reviews  of 
their  respective  system  portfolios  to  support  council 
decisions.  Flowever,  the  department  has  not  defined 
criteria  for  reviewing  defense  business  systems  at  an 
appropriate  level  in  the  department  based  on  factors 
such  as  complexity,  scope,  cost,  and  risk,  in  support  of 
the  certification  and  approval  process. 
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GAO  report,  recommendation,  Partially 

and  category  Implemented  implemented  GAO  assessment 


3.  The  Secretary  of  Defense  should  direct  X 

the  appropriate  DOD  management  entity 
to  develop  guidance  requiring  military 
departments  and  other  defense 
organizations  to  use  existing  business 
enterprise  architecture  content  to  more 
proactively  identify  potential  duplication 
and  overlap.  (Investment  Management) 


DOD  has  developed  guidance  requiring  military 
departments  and  other  defense  organizations  to  use 
existing  business  enterprise  architecture  content  to 
more  proactively  identify  duplication  and  overlap.  In 
particular,  the  department’s  April  2015  business 
enterprise  architecture  compliance  guidance  states 
that  examining  programs  for  potential  duplication  and 
overlap  should  occur  during  the  problem  statement 
requirements  analysis  process,  which  is  to  occur  early 
in  a  program’s  life  cycle.  In  addition,  the  department’s 
December  2014  problem  statement  requirements 
validation  guidance  calls  for  an  enterprise  architecture 
analysis  to  be  conducted  that  is  to  determine  if  a 
capability  already  exists  within  the  organization  or 
elsewhere  across  the  DOD.  If  a  solution  already  exists, 
the  problem  statement  sponsor  is  to  direct  that  the 
existing  solution  be  reused.  In  addition,  officials  from 
the  Office  of  the  DCMO  demonstrated  that  its  new 
Integrated  Business  Framework-Data  Alignment  Portal 
tool  can  be  leveraged  to  identify  potentially  duplicative 
systems  based  on  business  enterprise  architecture 
compliance  information  that  has  been  entered  into  the 
system. 


Source:  GAO  analysis  of  DOD  documentation  |  GAO-15-627. 

®GAO-1 3-557. 
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'  CHIEF  MANAGEMENT  OFFICER 

9010  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-9010 


June  30,  2015 


Director,  Information  Management  and  Technology  Resources  Issues 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Ms.  Cha: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability 
Office  (GAO)  draft  report  GAO-15-627,  “DoD  Business  Systems  Modernization:  Additional 
Action  Needed  to  Achieve  Intended  Outcomes,”  dated  June  5, 2015,  (GAO  Code  31 161 5).  The 
Department  concurs  with  the  recommendation.  Our  response  to  the  recommendation  is  provided 
in  the  attachment. 

The  Department  appreciates  the  opportunity  to  respond  to  your  draft  report.  We  look 
forward  to  your  continued  cooperation  and  dialog  toward  our  common  goal  of  improving 
defense  business  operations  and  achieving  cost  savings  resulting  from  successful  defense 
business  systems  programs  throughout  the  DoD. 

Should  you  have  any  questions,  please  contact  Mr.  Mark  Doehnert,  571-372-2959, 
Mark.P.Doehenrt.civ@mail.mil. 


Deputy  Chief  Management  Officer 

Enclosure: 

GAO  Draft  Report  dated  June  5, 2015 


Ms.  Carol  Cha 
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GAO  DRAFT  REPORT  DATED  JUNE  5, 2015 
GAO-15-627  (GAO  CODE  311615) 

“DOD  BUSINESS  SYSTEMS  MODERNIZATION:  ADDITIONAL  ACTION  NEEDED 
TO  ACHIEVE  INTENDED  OUTCOMES” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION; 

To  help  ensure  that  the  Department  can  better  achieve  business  process  reengineering  and 
enterprise  architecture  outcomes  and  benefits,  the  Government  Accountability  Office  (GAO) 
recommends  that  the  Secretary  of  Defense  utilize  the  results  of  GAO’s  portfolio  manager  survey 
to  determine  additional  actions  that  can  improve  the  Department’s  management  of  its  business 
process  reengineering  and  enterprise  architecture  activities. 

DoD  RESPONSE:  Concur.  The  Department  appreciates  the  survey  and  will  use  the  results  to 
help  make  improvements.  In  our  restructured  approach,  the  Department  will  focus  on  the 
following  improvements  to  our  practices: 

At  the  CMO  levels  of  both  the  military  departments  and  the  DoD,  the  focus  will  be  more 
on  management  of  functional  IT  portfolio  groups  rather  than  individual  systems.  This  change  in 
focus  is  being  communicated  particularly  to  the  OSD  staff,  who  are  following  the  trends  that 
have  been  practiced  in  the  military  departments. 

To  support  that  management  strategy,  and  consistent  with  comments  from  the  users,  the 
Department  will  restructure  the  BEA  to  focus  more  explicitly  on  the  business  processes  being 
executed  within  the  functional  domains,  spanning  all  levels  of  the  Department.  These  business 
process  descriptions  will  give  the  Department  a  solid  foundation  for  evaluating  overall 
improvements  to  Department  procedures  and  practices.  Where  IT  investments  are  a  possible 
solution  or  enabler  to  those  improvements,  the  same  architecture  work  will  serve  to  define 
requirements;  to  drive  business  reengineering  to  take  advantage  of  commercial  solutions  (while 
minimizing  customization);  and  to  make  program  investment  decisions.  At  the  OSD  level, 
updated  architecture  techniques  are  being  put  into  practice  with  staff  members  to  improve  skills. 

At  a  mechanical  level,  the  Department  is  reviewing  and  revising  how  the  workforce  at  all 
levels  can  access  the  BEA.  The  objective  is  to  make  information  more  readable  and 
discoverable,  as  conveyed  to  GAO  during  the  annual  assessment  this  year. 
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GAO  Contact 


Carol  R.  Cha,  (202)  512-4456  or  ChaC@gao.gov 
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GAO’S  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngc1@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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